An instruction-driven detector that turns any LLM managed on Amazon Bedrock into a configurable PII scanner. The detector uses prompts (not retraining) to define entities, runs on a swappable inference backend, and was evaluated against existing tools across public PII corpora.
PII detection is driven by instructions in the prompt, so adding or removing entity types requires a one-line edit, not retraining.
A uniform Inferencer interface makes the solution model-agnostic and deployable on Amazon Bedrock or in private infrastructure via a custom adapter.
Benchmarks on five public corpora across nine LLM-based detectors show the approach outperforming an off-the-shelf tool in the reported evaluation.
# What the project does This work describes a configurable, model-agnostic PII detector built around large language models. Instead of training a token-classification model with a fixed schema, the detector encodes the PII schema and output format into a system prompt. Any LLM that accepts a list of messages and returns assistant text can be used as the backend through a uniform Inferencer interface.
# Why this matters for teams PII appears in messy, multilingual free text (support transcripts, chat logs, HR records) and often includes domain-specific identifiers that fixed-label taggers miss. Traditional token-tagging models require relabeling and retraining to add new PII categories. This instruction-driven approach treats the schema as configuration: add, remove, or change entity types by editing the prompt, not by rebuilding models.
# How the detector is built (high level)
# Trade-offs and configuration choices
# Evaluation summary The detector was scored span-for-span alongside eight other LLM-based detectors, including the OpenAI PrivacyFilter, using a common ground truth across five public PII corpora. The Amazon Bedrock–based detector outperformed an off-the-shelf tool in that evaluation. The post includes repository code and a walkthrough for running the detector on your own data.
# Practical next steps for teams
# Where to find the implementation The detector is provided as the pii-detector package inside the sample-llm-pii-detection repository. The repository includes the prompt templates, Inferencer adapter examples, parsing modules, and an end-to-end runbook.
How LLM hallucination detection works: groundedness scoring, semantic entropy, judge models and fine-tuned detectors compared, with where each one fails.
These are the lessons we learned evaluating LLMs for real-world secret scanning. The post How to evaluate LLMs before production appeared first on The GitHub Blog .
by Thomas Sharpton, Edward W. Davis II, Alexandra Alexiev Responsible use of large language models (LLMs) in life science research demands a unified approach to risk, yet existing guidance treats prompting and verification as separate topics rather than as integrated components of a single risk management framework. Th
Background: Federated learning (FL) enables multi-institutional model training on clinical text without sharing raw data; however, gradient inversion methods can reconstruct sensitive information from shared model updates. The extent of such privacy leakage in FL applied to radiology reports, and the role of tokenizer
Learn how to automate end-to-end PII detection and redaction from scanned documents at scale using Amazon Bedrock Data Automation with a custom blueprint, AWS Step Functions, and AWS Lambda. A custom blueprint redacts sensitive fields with field-level precision, and a token matching quality check raises recall across d
Benchmark two 30B Mixture-of-Experts models, Qwen3-Coder-30B and NVIDIA Nemotron-3-Nano-30B, across G5, G6, G6e, and G7 GPU instances on Amazon SageMaker AI. Compare throughput, latency, and cost-per-token, and see how G7's NVIDIA Blackwell GPUs deliver measurable price-performance gains for real-time LLM inference.
Loading more related stories...
Open the app view to save this story, compare related coverage, and continue from the same source.
