Heuristic Models in Privacera Discovery¶

Privacera supports a broad range of heuristic models designed to identify and classify sensitive data using logic, pattern recognition, and validation mechanisms. These models help automate sensitive data tagging and improve classification accuracy.

1. Generic Models¶

These are baseline models with configurable parameters to guide pattern matching and logic-based detection.

2. Credit Card Model¶

Supported Default Credit Card Types: AMEX, MASTERCARD, VISA, DINERS, DISCOVER, JCBCARD, MAESTRO, CHINA_UNIONPAY, INSTAPAYMENT, MIR, RUPAY, TROY, VERVE, HIPERCARD, AURA, CARNET, BCGLOBAL

To support additional card types, use:

Regex property names

ADDITIONAL_REGEX_<CARD_TYPE>: Custom regex for matching a specific credit card type.

Examples of Additional Regexes:

1
2

ADDITIONAL_REGEX_JCB: ^((?:2131|1800|35\d{3})\d{11})$
ADDITIONAL_REGEX_MAESTRO: ^((?:5018|5020|5038|6304|6759|6761|6763)\d{8,15})$

3. Date of Birth (DOB) Model¶

4. EIN Model¶

5. Geo Latitude/Longitude Model¶

6. ITIN Model¶

7. MIME Model¶

Examples: EXEC_MIME_KEYWORD, IMAGE_MIME_KEYWORD

8. Phone Number Model¶

9. SSN Model¶

This model identifies possible SSNs (Social Security Numbers) in data by applying pattern matching, format checks, and optionally statistical/randomness-based validations.

There are two main phases:

A. Pattern Matching Phase

• Uses regular expressions (regex) to identify common SSN formats (e.g., 123-45-6789, 123456789, or 6789).
• Configurable using pattern-related flags like STRICT_PATTERN, USE_9_DIGIT_PATTERN, etc.

B. Statistical Validation Phase

• Applies advanced validations like entropy, digit distribution (variance), and sequentiality.
• These checks assess how "random" or "natural" the SSN-like string appears.
• Controlled by flags like USE_SEQUENCE_CHECK, USE_VARIANCE_CHECK, etc.
• All these are collectively controlled by USE_ADDITIONAL_VALIDATIONS.

Configuration Groups¶

A. Pattern Configuration

• Customize what qualifies as an SSN match:

B. Basic Validation Settings

• Check for basic data quality and known issues:
• Examples : "111111111", "987654321", "000000000", "1234567890", "12345678901"

C. Entropy Checks

• Entropy measures how random/unpredictable a number string is.

D. Variance Checks

• Variance looks at how digits are distributed — helps filter non-random digit repetition.

E. Sequence Checks

• Looks for sequential patterns

F. Sampling Control

• These determine when randomness-based logic should apply.

Examples of Invalid SSNs: - Starts with 9, 666, 000, or 98765432 - Middle digits = 00 - Last digits = 0000 - Dummy values: 123456789, 111111111, etc.

10. VIN Model¶

• Detects Vehicle Identification Numbers (VINs).
• Validates using length and VIN-specific checksum.

11. ZIP Code Model¶

12. Driving License Model¶

• Detects potential U.S. driver license numbers across 50 states.
• Utilizes state-specific regex patterns for structural validation.
• Applies robust heuristics to minimize false positives:
  • Length must be between 4–17 characters.
  • Rejects low-entropy or repetitive sequences.
  • Enforces character diversity to ensure validity.

13. Brazil CPF Model¶

The Brazil CPF (Cadastro de Pessoas Físicas) model identifies Brazilian individual taxpayer identification numbers using pattern matching and validation algorithms.

The model identifies CPF numbers based on the following criteria:

• Format: An 11-digit number in XXX.XXX.XXX-XX,XXXXXXXXXXX or XXX XXX XXX XX format.
• Validation: Uses the CPF checksum algorithm (mod 11).
• Exclusions: Automatically excludes known invalid patterns (e.g.,000.000.000-00, 111.111.111-11).

Custom Formats

You can define custom CPF formats using the ADDITIONAL_REGEX_<TYPE> property. Replace <TYPE> with a descriptive name for your custom format.

Examples

1
2

ADDITIONAL_REGEX_FORMATTED: ^((?:\d{3}\.\d{3}\.\d{3}-\d{2}))$
ADDITIONAL_REGEX_NUMERIC: ^((?:\d{11}))$

14. Brazil CNPJ Model¶

The Brazil CNPJ (Cadastro Nacional da Pessoa Jurídica) model identifies Brazilian company identification numbers using pattern matching and validation algorithms.

The model identifies CNPJ numbers based on the following criteria:

• Format: A 14-digit number in XX.XXX.XXX/XXXX-XX,XXXXXXXXXXXXXX or XX XXX XXX XXXX XX format.
• Validation: Uses the CNPJ checksum algorithm (mod 11).
• Exclusions: Automatically excludes known invalid patterns (e.g., 00.000.000/0000-00, 11.111.111/1111-11).

Custom Formats

You can define custom CNPJ formats using the ADDITIONAL_REGEX_<TYPE> property. Replace <TYPE> with a descriptive name for your custom format.

Examples

1
2

ADDITIONAL_REGEX_FORMATTED: ^((?:\d{2}\.\d{3}\.\d{3}\/\d{4}-\d{2}))$
ADDITIONAL_REGEX_NUMERIC: ^((?:\d{14}))$

15. Canada SIN Model¶

The Canada SIN (Social Insurance Number) model detects Canadian Social Insurance Numbers issued to individuals for tax and government benefit purposes. The model leverages pattern recognition and checksum validation logic to accurately identify valid SIN values while reducing false positives.

The model identifies Canada SIN numbers based on the following criteria:

• Format: Exactly nine digits in one of the following formats:

  Format	Example
  Hyphen-separated	123-456-789
  Space-separated	123 456 789
  Raw (no separator)	123456789

  Sequences of fewer than nine digits are neither matched nor padded to nine. Word boundaries prevent substrings of longer digit sequences (e.g. 10 or more digits) from being matched.

• Validation: The full nine-digit value is validated using the Luhn algorithm. The first digit must not be 0 or 8.

• Exclusions: Automatically excludes known invalid patterns, such as all-zero sequences and repetitive digits (e.g. 111-111-111).

Custom Formats

You can extend SIN detection to additional formats by defining custom regex patterns using the ADDITIONAL_REGEX_<TYPE> property. Replace <TYPE> with a descriptive name for your format.

Examples

1
2
3

ADDITIONAL_REGEX_FORMATTED: ^((?:\d{3}-\d{3}-\d{3}))$
ADDITIONAL_REGEX_SPACED: ^((?:\d{3}\s\d{3}\s\d{3}))$
ADDITIONAL_REGEX_NUMERIC: ^((?:\d{9}))$

16. Canada Postal Code Model¶

The Canada Postal Code model identifies Canadian postal codes using pattern matching and character-set validation. It uses the tag CANADA_POSTAL_CODE.

The model identifies Canada Postal Code values based on the following criteria:

• Format: Postal codes follow the pattern A1A 1A1 — a Forward Sortation Area (FSA) of letter-digit-letter, an optional space, then a Local Delivery Unit (LDU) of digit-letter-digit. Both A1A 1A1 (with space) and A1A1A1 (without space) are accepted.
• Keyword dictionary: The strict classification rule uses the CANADA_POSTAL_CODE_KEYWORD dictionary — which includes terms such as postal code, postcode, and code postal — alongside the detector to improve match confidence.

• Validation: Only permitted letters are accepted in each position:

  Position	Excluded Letters	Allowed Set
  1st (FSA, letter)	D, F, I, O, Q, U, W, Z	A, B, C, E, G, H, J–N, P, R, S, T, V, X, Y
  3rd (FSA, letter)	D, F, I, O, Q, U	A, B, C, E, G, H, J–N, P, R, S, T, V–Z
  6th (LDU, letter)	D, F, I, O, Q, U	A, B, C, E, G, H, J–N, P, R, S, T, V–Z

Examples

• Column or field names: Postal Code, Postcode, Code postal, Zip, Mailing Code
• Sample values: K1A 0B1, M5H 2N2, T2E 7W2 (with space); K1A0B1, M5H2N2 (no space)

17. Canada Driver License Model¶

• Detects potential Canadian driver license numbers across all 13 provinces and territories.
• Utilizes province-specific regex patterns for structural validation.
• Applies robust heuristics to minimize false positives:
  • Length must be between 5–17 characters.
  • Rejects sequential or repetitive patterns.

The model identifies Canada Driver License values based on the following criteria:

Valid values for EXCLUDED_PROVINCES (use exact names; case-insensitive):

ALBERTA, BRITISH_COLUMBIA, MANITOBA, MANITOBA_NO_HYPHEN, NEW_BRUNSWICK, NEWFOUNDLAND_AND_LABRADOR, NOVA_SCOTIA, ONTARIO, PRINCE_EDWARD_ISLAND, QUEBEC_HYPHEN, QUEBEC_SPACE, QUEBEC_NO_SEP, SASKATCHEWAN, NORTHWEST_TERRITORIES, NUNAVUT, YUKON

Conclusion¶

Privacera Discovery's model-based classification supports numerous sensitive data formats through specialized validation rules and pattern matching. These models help achieve consistent and automated tagging of data across domains such as finance, healthcare, HR, and more.