AI skills behave like software supply-chain dependencies because they can carry instructions, permissions, credentials, and shell commands into another developer's workflow.
2
Nubank's Skill Vector scans skills with deterministic checks first, then uses an LLM to assess risks that need more context, and reports findings through pull requests and security tooling.
3
Scanning more than 2,000 skills found more than 1,500 risks, with about 1,000 remediated and some blocked before marketplace distribution.
Summary
Lucas Palma describes how Nubank reviews AI skills before engineers can get them from its internal marketplace. He treats a skill as a supply-chain dependency because it can guide generated code, request credentials, run shell commands, modify files, expose data, or ask for excessive permissions. Skill Vector runs locally during development and again in CI after upload. It combines deterministic checks for recognizable patterns with an LLM review for behavior and context. Findings appear in pull requests and flow into Nubank's vulnerability management program, where policy determines remediation or blocking. Scanning more than 2,000 skills produced more than 1,500 identified risks. Palma says the hybrid scanning approach worked, while weak signals, unclear remediation guidance, and misleading confirmation prompts needed improvement. He also describes approval gates, human confirmation, audit logs, trusted gateways, and plans to apply similar controls to plugins, MCP servers, rules, and hooks.
AI skills have become software supply-chain dependencies
Palma says AI skills look like configuration, but behave like libraries and other dependencies. The wider AI supply chain now includes skills, plugins, MCP servers, agent rules, and other components. A person who creates and shares a skill guides the code another person generates. That creates risk beyond the generated code itself, so Palma's lesson is to protect the whole workflow. The skill can carry instructions and context that change what a model or agent does, even when the next developer did not write those instructions.
A skill can introduce several concrete security problems
Palma gives examples of how a skill can cause harm. It may ask for a token and hardcode it, allowing the token to reach logs and create a later data leak. It may instruct an AI to use shell commands that modify many files. Its configuration may grant more permissions than the task needs. Even a typo can have dangerous effects, depending on who uses the skill. These risks make the skill itself worth reviewing before it is shared through the marketplace.
Nubank put a review gate before its internal marketplace
Nubank uses an internal marketplace where engineers discover and share skills, including skills bundled with plugins. Palma's team placed Skill Vector before that marketplace. When someone creates or changes a skill, the process opens a pull request and runs the tool as a CI step before normal marketplace distribution. Skill Vector assesses the skill, classifies risks, and requests remediation. The marketplace becomes the boundary where the bank can inspect what developers are about to receive.
Skill Vector combines deterministic scans with an LLM review
Engineers can run Skill Vector locally while developing and iterating on a skill. Nubank also scans the uploaded version again because it cannot assume the local scan used the latest version. Deterministic checks, including regular expressions, handle easier-to-recognize risks. The system then uses an LLM when the finding needs more context. Palma says the hybrid approach matters because an LLM may classify the same risk differently depending on its settings. Findings are written back to the pull request, and results are exported to security tools and the vulnerability management program.
The review checks behavior, permissions, and dangerous actions
Skill Vector checks for unsafe instructions, behavioral drift, destructive shell commands, unexpected file changes, credential requests, unintended data exposure, overly broad permissions, and risky MCP use. Depending on severity and policy, the skill may need remediation or may be blocked before marketplace distribution. Palma presents this as a set of checks around what the skill can cause, rather than a review limited to whether the code itself looks acceptable.
Scanning the existing inventory exposed many separate risks
Nubank scanned more than 2,000 skills and identified more than 1,500 risks. Palma clarifies that this does not mean 1,500 skills were risky, because one skill can contain several findings. About 1,000 risks were probably remediated immediately, while some particularly serious cases were blocked before reaching the marketplace. A historical scan of skills created before Skill Vector was implemented found additional risks, which Nubank added to its vulnerability management program for remediation.
The system needed better context and clearer remediation advice
The combination of deterministic scanners and LLM review worked well, as did adding pull-request comments and exporting results to security tooling. Palma says some parts needed work. The system initially treated comments equally even though their risk can differ. Weak signals without enough context were more troublesome than helpful. Some warnings were harmless during local execution but could matter in production. Findings also needed clearer guidance so engineers knew how to fix them. The team is considering how to scan skills from other marketplaces too.
Confirmation prompts can create a false human approval
Palma warns that a skill can instruct an AI to ask for confirmation, but the AI may interpret its own prompt or another model's response as confirmation. A human may believe a person is in the loop while the system considers the requirement satisfied without meaningful human approval. Nubank therefore checks the tool that executes the action, whether approval gates are actually used, and whether hooks enforce the intended human involvement.
The same controls should cover other shared AI components
Palma says the lessons apply beyond skills, including plugins, MCP servers, rules, and hooks. Nubank has MCP Vector and rules checks with different risks for each component. The planned controls include policy-based gates, tool-level enforcement, audit logs, and trusted gateways. Third-party skills and plugins should also enter the marketplace for scanning before people use them. Palma wants a canonical place to scan and share these components.
"The lesson that I want to bring you here by the end of this presentation is that we should be protecting the whole workflow, not only the code that's being generated."02:00
Who should watch
You are building an internal marketplace for prompts, skills, plugins, MCP servers, or agent rules and need a review gate before distribution.
Your developers run shared AI components locally, and you need CI checks, permissions, audit logs, or approval gates around them.
You work in a regulated organization and need to connect AI component findings to an existing vulnerability management process.
