bioptimus.observability.anonymize

PII anonymization for logs and support-bundle artifacts. Whole-slide-image paths, filesystem paths, and object-store URIs frequently embed patient identifiers (for example TCGA-A6-1234_patientX.svs). Before any log line, manifest, or bundle component leaves the customer environment it is passed through this module. By default every recognized field (ANONYMIZED_FIELDS) is replaced by a stable hash, so a given slide keeps a consistent pseudonym across every artifact and “which slide failed” correlation survives. The digest is unsalted and truncated by design, so this is pseudonymization for support triage, not cryptographic anonymity: a party who can enumerate the identifier space can confirm a value (see ADR-005). The per-field policy is configurable via AnonymizationConfig: a field can instead be redacted entirely or kept verbatim. Paths and URIs embedded in free text are always redacted. Nothing is ever transmitted automatically — the helpers only rewrite values in place. Entry points:

AnonymizationConfig, the policy controlling each field’s Strategy (DEFAULT_CONFIG hashes every recognized field);
AnonymizingFilter, a logging.Filter that scrubs log records in place; and
the pure helpers hash_identifier, redact_text, scrub_user_paths, anonymize_value, and anonymize_mapping, reused by the support-bundle builder to scrub files on disk.

Warnings: This redaction is best-effort, not a guarantee. It recognizes a fixed set of field names and path/URI shapes; PII in an unrecognized field, an unusual identifier format, or an unexpected free-text form can pass through unredacted. The hash used is unsalted, truncated pseudonymization, not cryptographic anonymity. Bioptimus makes no warranty that the output is free of PII and accepts no liability for it. Audit the redacted output internally before sharing it outside your environment.

REDACTION_MARKER

Replacement token substituted for every redacted value.

USER_PATH_MARKER

Placeholder substituted for the user name in a home-directory path, so a diagnostic path keeps its structure (for example .venv/bin/python3) without exposing whose account it is.

HASH_PREFIX

Prefix marking a value that has been replaced by a one-way hash pseudonym.

HASH_FIELDS

Identifier fields that act as correlation keys. Hashing (rather than redacting) these preserves the “which slide failed” link across every log line, manifest, and ledger entry without directly exposing the underlying identity (though the unsalted, truncated hash is pseudonymization, not cryptographic anonymity).

PII_FIELDS

Path-like fields. Their values embed directory structure and user names beyond the bare identity; hashing keeps them correlatable while removing the layout. Configure them to Strategy.REDACT to drop them instead.

ANONYMIZED_FIELDS

Every field anonymized by default. Each is hashed unless an AnonymizationConfig override selects a different strategy.

Strategy

class Strategy(str, Enum)

How a recognized field’s value is anonymized.

HASH

Replace with a stable one-way hash, preserving cross-artifact correlation.

REDACT

Replace with REDACTION_MARKER, removing the value entirely.

KEEP

Keep verbatim; only embedded paths, URIs, and user names are scrubbed.

AnonymizationConfig

@dataclass(frozen=True)
class AnonymizationConfig()

Per-field anonymization policy. The default strategy applies to every field in ANONYMIZED_FIELDS; overrides selects a different strategy for named fields. DEFAULT_CONFIG hashes every recognized field, so all identifiers and paths become stable pseudonyms while remaining correlatable. Attributes:

default - Strategy applied to recognized fields without an override.
overrides - Field-name to strategy mapping that wins over default.

strategy_for

def strategy_for(key: str) -> Strategy

Returns the strategy for key.

The field name being anonymized.

Returns: The override for key if one exists, the default strategy for any recognized field, otherwise Strategy.KEEP.

DEFAULT_CONFIG

Default policy: hash every recognized field.

redact_text

def redact_text(text: str) -> str

Redacts paths, URIs, and slide file names from a free-text string. Replaces any path or bare file name ending in a known whole-slide-image extension, plus generic absolute paths and s3:///file:// URIs, with REDACTION_MARKER. Non-matching text is returned unchanged.

Arbitrary text that may embed slide paths or file names.

Returns: The text with slide path and file-name occurrences redacted.

scrub_user_paths

def scrub_user_paths(text: str) -> str

Masks the user name embedded in home-directory paths. Replaces the account component of a /home/<name> or /Users/<name> path with USER_PATH_MARKER, leaving the rest of the path intact so its diagnostic value (for example a virtual-environment or project layout) survives. /root and paths outside a home directory are left unchanged.

Arbitrary text that may embed home-directory paths, for example an interpreter path or a temp-directory location.

Returns: The text with the user name in any home-directory path masked.

hash_identifier

def hash_identifier(value: object) -> str

Returns a stable pseudonym for an identifier value. Hashes the string form of value with SHA-256 and returns HASH_PREFIX followed by the leading digest characters. Values that already match the SDK pseudonym shape are returned unchanged, making anonymization idempotent when logs were redacted at emission and again at export. The result is deterministic, so the same identifier maps to the same pseudonym across every log line and bundle artifact, preserving the “which slide failed” correlation. The digest is unsalted and truncated by design (stable pseudonyms, short log lines), so it is pseudonymization for support triage, not cryptographic anonymity: a party who can enumerate the identifier space can confirm a value (see ADR-005).

The identifier to pseudonymize, for example a slide or patient ID. Non-string values are hashed by their str form.

Returns: A pseudonym of the form "sha256:<hex>".

anonymize_value

def anonymize_value(key: str,
                    value: Any,
                    config: AnonymizationConfig = DEFAULT_CONFIG) -> Any

Anonymizes a single key/value pair. The strategy for key is resolved from config: a recognized field is hashed (hash_identifier) or redacted, and a kept field keeps its value with embedded slide paths, absolute paths, URIs, and home-directory user names scrubbed. A None value is never hashed.

The field name associated with value.

The field value to inspect.

The anonymization policy. Defaults to DEFAULT_CONFIG, which hashes every recognized field.

Returns: The hashed, redacted, or scrubbed value depending on the strategy.

anonymize_mapping

def anonymize_mapping(
        data: Mapping[str, Any],
        config: AnonymizationConfig = DEFAULT_CONFIG) -> dict[str, Any]

Recursively anonymizes a mapping. Applies anonymize_value to every entry, recursing into nested mappings and sequences. The input is not mutated.

The mapping to anonymize, for example a manifest or ledger.

The anonymization policy. Defaults to DEFAULT_CONFIG.

Returns: A new mapping with PII values anonymized.

AnonymizingFilter

class AnonymizingFilter(config: AnonymizationConfig = DEFAULT_CONFIG, name: str = '')

A logging filter that anonymizes PII in records in place. Scrubs the structured-field payload attached by bioptimus.observability.logging.log_event, the record message, and any positional message arguments, so PII never reaches a handler’s formatter. Recognized fields are hashed or redacted per the configured AnonymizationConfig. The filter always returns True — it never drops records.

filter

def filter(record: logging.LogRecord) -> bool

Anonymizes PII in the given record.

The log record to scrub. It is mutated in place.

Returns: True always, so the record continues through the handler.

Overview

Get Started

Preprocessing

Workflows

Troubleshooting

Reference

REDACTION_MARKER

USER_PATH_MARKER

HASH_PREFIX

HASH_FIELDS

PII_FIELDS

ANONYMIZED_FIELDS

Strategy

HASH

REDACT

KEEP

AnonymizationConfig

strategy_for

DEFAULT_CONFIG

redact_text

scrub_user_paths

hash_identifier

anonymize_value

anonymize_mapping

AnonymizingFilter

filter

​REDACTION_MARKER

​USER_PATH_MARKER

​HASH_PREFIX

​HASH_FIELDS

​PII_FIELDS

​ANONYMIZED_FIELDS

​Strategy

​HASH

​REDACT

​KEEP

​AnonymizationConfig

​strategy_for

​DEFAULT_CONFIG

​redact_text

​scrub_user_paths

​hash_identifier

​anonymize_value

​anonymize_mapping

​AnonymizingFilter

​filter

REDACTION_MARKER

USER_PATH_MARKER

HASH_PREFIX

HASH_FIELDS

PII_FIELDS

ANONYMIZED_FIELDS

Strategy

HASH

REDACT

KEEP

AnonymizationConfig

strategy_for

DEFAULT_CONFIG

redact_text

scrub_user_paths

hash_identifier

anonymize_value

anonymize_mapping

AnonymizingFilter

filter