# LLM Safeguards: Security, Privacy, Compliance, and Anti-Hallucination

Daniel Whitenack, Prediction Guard | AI Engineer World's Fair 2024 | 34:10

Source: https://www.youtube.com/watch?v=jdeMJJ_oNYg
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/llm-safeguards-security-privacy-compliance-and-anti-hallucination
Published: 2024-12-31
Tags: guardrails, human-in-the-loop, privacy, security

## TL;DR
- Enterprise LLM applications need safeguards for hallucination, supply-chain attacks, vulnerable model servers, data leaks, and prompt injection.
- Grounding an answer with retrieved data does not prove that the output is factually consistent, so Daniel recommends a separate factual-consistency model or ensemble.
- Security controls should include trusted model registries, endpoint monitoring, privacy filters, confidential computing, prompt-injection detection, and restricted permissions for agents.

## Summary
Daniel Whitenack presents a practical checklist for deploying open-access LLMs in enterprise systems. He begins with the risks that appear around an ordinary model API: hallucinated answers, malicious model assets, exposed or poorly operated servers, sensitive data in prompts, and prompt injection. Retrieval-augmented generation can ground an answer in company data, but it also creates privacy and access-control problems. Daniel describes using factual-consistency models to compare generated text with ground truth and produce a score. He recommends curated model registries, hash checks, endpoint monitoring, private-data filters, confidential computing, and classification models for prompt injection. The talk also covers latency, pre-production testing, data permissions, SIEM signals, and agent permissions. His advice is grounded in ordinary enterprise security practice: reuse controls that already work, make them visible and configurable, and avoid giving an agent more authority than it needs.

## Key ideas
### Enterprise AI adoption often starts with risk management rather than copilots
[00:03](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=3s)
Daniel frames the talk around the difference between AI's promise and what adoption looks like inside real companies. He focuses on open-access large language models because many enterprises are considering them as part of their AI strategy, even when they are not using them for every application. His team has worked with customers on deploying secure and accurate AI systems, and he turns those lessons into a checklist. The checklist covers problems that can appear when a user sends input to a model and receives generated output, including inaccurate answers, attacks on model infrastructure, privacy failures, and malicious instructions.

### Hallucinations become liability risks when models support high-stakes work
[03:41](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=221s)
LLMs generate text from patterns that may reflect the internet, including outdated or inaccurate information. Daniel gives the example of asking for the health benefits of eating glass, where the model may confidently produce an answer even though the premise is dangerous. He describes a customer application that assists field medics in disaster-relief and military situations, where one situation may involve 16 casualties. A wrong answer in that setting can create serious consequences. Even lower-stakes enterprise uses can bring liability concerns when a model confidently gives incorrect information.

### Retrieval grounds outputs but does not show that they are correct
[10:51](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=651s)
A common response to hallucination is to retrieve a document and insert its contents into the prompt. This can make a summarization follow the source email, but Daniel says the approach can fail badly after appearing to work most of the time. It also introduces privacy concerns because the application is now sending company data into the model. His team compares the generated output with the ground-truth text using models trained to detect factual consistency. They use an ensemble of such models to produce a score. Daniel names UniEval and BARTScore as examples from the research literature, and also mentions an LLM-as-judge approach.

### Model assets need the same supply-chain discipline as other open-source code
[05:30](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=330s)
An open model is not just a file that runs by itself. It may come with code, packages, and third-party dependencies, creating a supply-chain path for malicious code. Daniel recommends pulling models from a trusted registry, checking that the license fits the use case, and verifying a hash or attestation when obtaining assets from a third party. Teams can also copy approved models into their own Hugging Face repository or internal registry. He compares automated downloading of model code with automatically finding and running arbitrary GitHub code, which most security teams would reject. He also advises keeping Transformers' untrusted-code option disabled.

### Model servers need ordinary endpoint and operational security
[15:43](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=943s)
Whether a model runs on a company's own infrastructure or through a private tenant, it is still an API service on a server. Daniel recommends familiar controls such as endpoint monitoring, file-integrity monitoring, penetration tests, and red-team exercises. Teams should ask providers where models run and what security evidence they can provide about those servers. He points out that data scientists may not have the same experience as distributed-systems or microservices teams with running these services at scale. Capacity and resilience therefore belong in the deployment plan alongside model quality.

### Privacy filters can block, remove, or replace sensitive information
[17:10](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=1030s)
Retrieval systems can place personally identifiable information into a prompt, such as an email address, home location, or details from a support ticket. The model can then expose that information in its response. Daniel recommends putting a detector in front of the model for PII and more specialised data such as protected health information. Depending on the application, the filter can block the request, strip out the sensitive fields, or replace them with fake values. These controls do not solve every server-side exposure, since prompts may still be logged or held in unencrypted memory.

### Confidential computing protects data while requests are processed
[18:27](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=1107s)
Filtering data before an LLM call is useful, but Daniel also considers what happens to prompts inside the model environment. He points to confidential-computing methods that encrypt server memory, including Intel SGX and TDX, as well as trusted attestation services that verify a server environment before a request is sent. These controls address a different part of the problem from input filtering. They are relevant when prompts contain information that should not be visible to operators or attackers who gain access to the model server.

### Prompt-injection defenses should be configurable and visible
[19:20](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=1160s)
A malicious user can place instructions in a prompt that tell the model to ignore its rules and reveal a server IP or private company information. The risk grows when an application connects the model to a knowledge base, database, or tools. Daniel describes a safeguard layer built from examples of known and newly observed injections plus an ensemble of classification models. The system can filter likely attacks at the front end. He also wants teams to inspect the model's scores and change thresholds, rather than receiving an unexplained moderation decision. He later describes calling the injection model separately for pre-production testing.

### Agent permissions should be restricted and reviewed through dry runs
[31:31](https://www.youtube.com/watch?v=jdeMJJ_oNYg&t=1891s)
Agents create an additional risk because they can take actions with permissions granted by the surrounding application. Daniel warns against giving an agent too much authority too early. An assistant that changes computer settings may need administrator access, and hallucinations could then cause unmanaged changes. One response is to restrict permissions. Another is to have the agent produce a dry run of its intended actions for a person to approve or edit. Daniel says this pattern is useful for tasks such as generating network configuration, because a skilled operator can review and modify the proposed changes faster than creating them from scratch.

## Notable quotes
- "The problem with this is that one, it stresses one of our other problems, which is the data breach privacy problem." (11:41)
- "People don't just sort of do an automated search of GitHub for code to do this and then automatically pull it down and run it." (14:54)
- "You want to think about these additions around the model not as additional LLM calls if possible, but as creative NLP or vector type of operations when you can." (23:40)
- "One way to deal with that is to restrict permission. Another way to deal with that is to have a dry run of what the agent is going to do and then have that approved." (32:46)

## Tools & references mentioned
- Prediction Guard
- Transformers
- Hugging Face
- UniEval
- BARTScore
- LLM-as-judge
- Intel SGX
- Intel TDX
- Rebuff
- PostgreSQL
- pgvector
- Amazon S3
- SOC 2
- SIEM
- OWASP Top 10

## Who should watch
- You are putting an open model behind company data and need a concrete list of risks before production.
- Your RAG system produces plausible answers, but you have no separate way to measure factual consistency or explain blocked prompts.
- You are adding tools or agent actions and need to decide which permissions require restriction, review, or human approval.

## Related talks

- [Building security around ML](https://aietalks.com/talks/building-security-around-ml) (Dr. Andrew Davis, HiddenLayer, 25:01)
- [Navigating Challenges and Technical Debt in LLMs Deployment](https://aietalks.com/talks/navigating-challenges-and-technical-debt-in-llms-deployment) (Ahmed Menshawy, Mastercard, 16:15)
- [Trust, but Verify](https://aietalks.com/talks/trust-but-verify) (Shreya Rajpal, Guardrails AI, 19:41)
- [Your LLM Stack Is a 2008 Database With Better Marketing](https://aietalks.com/talks/your-llm-stack-is-a-2008-database-with-better-marketing) (Lovina Dmello, NVIDIA, 20:36)
- [The LLM Triangle: Engineering Principles for Robust AI Applications](https://aietalks.com/talks/the-llm-triangle-engineering-principles-for-robust-ai-applications) (Almog Baku, 26:19)