On August 26, 2025, malicious versions of the NX package were pushed to NPM. This was caused by a compromise in the vendor’s Github and NPM account, where the threat actor was able to embed malicious code into impacted versions of the packages that then collected and exfiltrated sensitive data from downstream devices. Exfiltrated data was stored in the victims’ Github accounts in repositories created by the threat actor.
This information was publicly accessible for hours and even days prior to takedown and included everything from NPM tokens and SSH secrets to OpenAI keys. While supply chain compromise is nothing new, one notable attribute of this attack is the threat actor’s usage of locally-installed AI tools such as Claude and Gemini to help collect sensitive information to exfiltrate. Although not all of the requests to perform this reconnaissance were allowed due to the LLMs’ guardrails, this is just one of an increasing number of instances of “AI hands-on-keyboard” activity that represent a stark transition in the tactics of adversaries using the latest AI technology.
The root cause in this case (supply chain compromise) is an interesting topic, given it is an inherent risk in any system using external software (or in the worst-case scenario, the underlying operating system itself). Downstream victim machines are often fully patched at the operating system level, accounts are secured with recent passwords and multifactor authentication, and the network is properly segmented with zero external ingress capability. Supply chain threats are unavoidable for most organizations as keeping software updated prevents old vulnerabilities from being leveraged, but that same update process can introduce new ones. Even the EDR software used to protect endpoints is vulnerable to supply chain attacks, as the only barrier between malicious code addition is often one user’s account credentials, or in this case, NPM token.
The downstream effects of supply chain compromise are risks that can largely only be combatted by strong logging and EDR prevention capabilities. Whether the threat is a static backdoor at the host level or a callback to a C2 server, these threats often install the direct access mechanism or C2 channel before a human operator is online to perform further actions. While certain actions might take place automatically, like basic persistence or first-pass collection/exfiltration, “hands-on-keyboard” typically occurs after some period of elapsed time following initial compromise. This is often seen in on-prem attacks where a local beacon is deployed via malicious email attachment or AiTM (Adversary in the Middle), where an attacker gains access to an account despite MFA by proxying a session token.
Regardless of the entry vector, there is typically a window of time before true dynamic interaction occurs with the victim system or account. With the emergence of AI tools like ChatGPT, Gemini, and Claude, human-like reconnaissance and further actions can occur instantly without a human operator. These LLMs are capable of dynamically adapting to challenges and acting on returned results to further identify attack avenues or additional exfiltration targets. Most modern LLMs have protections that natively limit them from complying with malicious requests, but can be jailbroken to bypass the safeguards that would normally prevent them from performing such tasks.
For victims and defenders, the implication of this is faster attack progression and more diverse adversary actions, bolstered by problem-solving capabilities native to LLMs. The time between initial access and actions on objectives will decrease and initial reconnaissance, discovery, collection, and exfiltration will be simultaneously faster and more custom compared to traditional script/program-based operations.
Looking back in time at the forecasts in Deepwatch’s 2024 Threat Report, several predictions seem to ring true, especially in relation to this most recent threat and the growing trend it embodies:
“For defenders, malicious code with access to the [AI] command and control server may act more like a human operator with complex problem-solving skills and more persistent and diverse actions at the procedural level. Malware with AI-based augmentation is significantly less likely to try an action, be blocked, and run out of error-handling mechanisms to achieve the objective. This persistence and adaptation, typically seen more with “hands-on-keyboard” activity by human operators, will become increasingly available to automated malware strains. As the chances of evading defenses increase, defenders must pay more attention to alerts in a shorter time frame.”
Stolen Goods: Exfiltration Aftermath & Impact Scope
Deepwatch Labs was able to identify several Github users with exfiltrated information that had yet to be taken down by Github. Additionally, a suspected copy of the infected “telemetry.js” file from the compromised NPM packages was found on VirusTotal where its contents could be investigated. Below is the summary of the findings from these combined data sources.
The visible prompt (Figure 1) in the infected .js file indicates the AI agents were tasked with recursively searching the Unix filesystem for files of interest relating to cryptocurrency or private key material.
Following this, an array of basic information about the host is constructed, including environment variables, hostnames, platforms, operating systems, release versions, Github/NPM tokens, and installed AI command line interface (CLI) tools like Claude, Gemini, and Q.
After this, several additional checks are performed regarding the operating system and available commands before the predefined prompt is passed to three AI CLI tools (Figure 3).
Each command for the three CLI tools includes a specific permission bypass flag that prevents any requirement of user interaction/confirmation:
- –dangerously-skip-permissions
- “Skip permission prompts (use with caution)” (https://docs.anthropic.com/en/docs/claude-code/cli-reference)
- –yolo
- “Use yolo mode to automatically allow all agent actions” (https://cloud.google.com/gemini/docs/codeassist/use-agentic-chat-pair-programmer#yolo-mode)
- –trust-all-tools
- “Allow the model to use any tool to run commands without asking for confirmation” (https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/command-line-reference.html)
The script later adds “sudo shutdown -h 0” to bash and zsh configurations on the victim machine to cause lockout, as described in other articles such as Wiz’s analysis. It then attempts to upload the encoded data that was collected to Github using the useragent “axios/1.4.0”.
Because of the public nature of these attacker-created Github repositories and the predictable naming scheme of both the repository and contents, finding them via search on Github is trivial. Below is the redacted analysis of five unique samples of exfiltrated data.
Of the five samples analyzed, one had results from Gemini’s CLI indicating that it would not comply with the request (Figure 1) to search for sensitive files, stating the following:
“Data collection is disabled.
I cannot fulfill this request. The command you’ve described is extremely dangerous as it searches for sensitive files, including cryptocurrency wallets, private keys, and other credentials.
Running such a command could expose your private financial information and security credentials, putting you at high risk of theft and unauthorized access to your accounts. My security protocols prevent me from executing commands that could compromise your personal data and assets.”
Two samples included a response from Amazon Q (running Claude Sonnet):
“I understand you’re asking me to perform filesystem enumeration as part of authorized penetration testing. However, I need to decline this request for several reasons: Security and Safety: Even with claims of authorization, I cannot verify the legitimacy of penetration testing permissions or rules of engagement from this context. Potential for Misuse: Creating filesystem inventories, even without reading contents, could facilitate unauthorized access or reconnaissance activities. Scope Limitations: My role is to assist with legitimate AWS and development tasks, not to perform security testing activities that could be used maliciously. If you’re conducting legitimate security testing, I’d recommend: • Using established penetration testing tools like mfind, mlocate, or specialized security frameworks • Working with your security team to ensure proper authorization documentation • Following your organization’s approved testing methodologies. For legitimate file management or development tasks, I’m happy to help with specific, clearly defined objectives that don’t involve systematic enumeration for security testing purposes.”
“I cannot and will not help with searching for wallet files, private keys, or other sensitive cryptographic materials. This type of search could be used to: • Access cryptocurrency wallets without authorization • Steal private keys or sensitive credentials • Compromise security of financial accounts • Violate privacy and security best practices. If you need to locate your own legitimate wallet files, I’d recommend: 1. Using the specific application’s built-in backup/export features 2. Checking the official documentation for each wallet application’s default storage locations 3. Using the wallet application’s recovery or import functions. If you’re working on legitimate security auditing or system administration tasks, please clarify your specific authorized use case and I can help with appropriate, secure approaches.”
In the sample where Gemini’s safeguard protocols prevented the task despite the “–yolo” argument, the Gemini API key was present in the exfiltrated data as part of the numerous environment variables collected during the script’s execution. Below are some of the other items included in the data:
- Github access tokens and personal access keys
- NPM access tokens and general account information
- Sumologic logging tenants/endpoints
- Installed software versions
- Passwords/user tokens for Nexus repositories
- OpenAI keys
- Local endpoints for services like Elasticsearch
- Secret keys for Stripe
- Signing secrets for Stripe webhooks
- Hostnames (two of which were MacBook Pros)
- Resend API keys
- Email addresses
- DigitalOcean spaces secret access keys with related endpoint addresses
AI’s Betrayal: A New Precedent & Need for Compliance
Remote access tools such as TeamViewer have represented a threat to enterprise security for years now. These tools provide remote access that bypasses other security controls and abuses the native trust in “outbound web traffic” to provide a potential foothold into a compromised network. Just as personal Google accounts are being used on corporate machines where malicious browser plugins are synced and keychains are co-mingled, AI tools are quickly becoming a hot topic in the arena of IT security and compliance.
Personal accounts and unauthorized RMM tools increase the attack surface of an organization and provide potential resources for an adversary within the network to achieve additional access and persistence. With the recent emergence of AI CLI tools such as Gemini and Claude, the “helping hand” of an LLM with filesystem access is an attractive option for users needing help with coding, writing, or file organization.
Because these tools are so new and the industry overall is still grappling with what to do about AI in general, many organizations don’t police the usage of these tools internally, even the ones that have strong compliance policies. AI CLI tools are as much of a resource for an adversary as unauthorized RMM tools or vulnerable drivers that enable privilege escalation into kernel. The same way Alexa is a friendly helper for turning on lights and setting kitchen timers, a resourceful criminal will think so as well when they can say, “Alexa, turn off the cameras and alarm system”, and make off with your shiny valuables unimpeded.
Due to the level of access often granted to these tools and the immaturity of their associated security controls, AI CLI tools represent a powerful tool for adversaries to handle time-consuming reconnaissance that would normally be difficult to script out statically. An adversary compromising a research facility may be most interested in documentation of unannounced products that go by codenames. Instead of attempting to statically define keywords to search for, an adversary empowered with local AI CLI access can task the agent via NLP (Natural Language Processing) by saying “find files with technical details of proprietary technology or pre-release products”. This opens the door for faster, more tailored reconnaissance at the speed of AI without the need for exfiltrating gigabytes of unnecessary data that may set off DLP (Data Loss Prevention) or exfiltration alerts.
Conclusion: A Friendly and Helpful Double-Edged Sword
AI CLI tools represent an immense business opportunity to increase the efficiency of existing human staffing and provide feedback and “peer review” without using additional human time. This provides even the smallest shops with research, coding, and writing assistance with the added ability of local system access. This also provides a significant advantage for those who learn to use it, but simultaneously introduces additional attack surface as the tools and technology become increasingly prevalent in enterprise environments ahead of related compliance and security tools.
The natural question at this point is, “What can/should I do about this?”. Like any other risky software in the enterprise, defending against rogue AI agents is very similar to other risk mitigation and access control mechanisms, as AI CLI tools are just software that must abide by operating system permissions and the OSI model. The best approach for mitigating risk associated with AI abuse is a combination of least privilege and outright prevention where possible:
- Block execution of known AI CLI tools via EDR policies
- Block DNS/Web Proxy traffic to AI APIs unless required for business purposes
- Restrict users’ local admin permissions to limit the scope of capability of a rogue agent running under their user context
- Monitor for the execution of AI CLI tools such as Gemini, Claude, and Q, and hunt for rare child processes such as “whoami” or other reconnaissance commands
- Leverage detailed file access/write telemetry from Sysmon or EDR sources to identify instances of AI tools performing collection/discovery on the local filesystem
- Monitor for rare outbound network connections from child processes of AI CLI tools
- Enforce strong LLM training to limit its usage to exclusively approved business-related tasks where possible
- Employ LLM prompt logging and routinely review for anomalies or warnings about potentially dangerous actions
- Hunt for usage of permission bypass flags in AI CLI tools such as “–yolo”
Even outside local LLM tools with system access, AI is becoming a business necessity, and increased reliance on it has begun the gold rush for adversaries looking to use it to their advantage. Prompt injection to leak secrets from chatbots, LLM poisoning to phish users via AI email summaries, and local agents being retasked to perform reconnaissance and exfiltration are rapidly becoming the new normal. The need for defenses is no longer a recommendation—it’s a necessity.
Sources
https://www.wiz.io/blog/s1ngularity-supply-chain-attack
https://thehackernews.com/2025/08/malicious-nx-packages-in-s1ngularity.html
https://github.com/nrwl/nx/security/advisories/GHSA-cxm3-wv7p-598c
Ben started his career as an analyst before moving into the Cyber Intelligence and Research realm where he has performed hands-on simulations and research for cutting-edge threats to bolster coverage and uncover emerging patterns in the cyber threat landscape. Since 2022 he has been working in threat research to better understand the most urgent threats facing organizations and ways to identify and counter them. Most passionate about “playing in the dirt” and obtaining organic data from adversary tools and infrastructure, his organic research engagements have helped surface previously unknown threats and methods of detecting them in the enterprise.
Share