# pointblank

> Find out if your data is what you think it is.

## Docs

### API Reference

#### Validate
> When performing data validation, use the `Validate` class to get the process started. It takes the target table and options for metadata and failure thresholds (using the `Thresholds` class or shorthands). The `Validate` class has numerous methods for defining validation steps and for obtaining post-interrogation metrics and data.


- [Validate](reference/Validate.html): Workflow for defining a set of validations on a table and interrogating for results
- [Thresholds](reference/Thresholds.html): Definition of threshold values
- [Actions](reference/Actions.html): Definition of action values
- [FinalActions](reference/FinalActions.html): Define actions to be taken after validation is complete
- [Schema](reference/Schema.html): Definition of a schema object
- [DraftValidation](reference/DraftValidation.html): Draft a validation plan for a given table using an LLM

#### Contracts and Pipelines
> Use `Contract` and `Step` to define declarative data quality contracts that specify what valid data looks like. Use `Pipeline` to enforce contracts at both boundaries of a data transformation (source and target), producing a `PipelineResult` with full introspection into what passed and what failed.


- [Contract](reference/Contract.html): A declarative boundary contract for pipeline data
- [Step](reference/Step.html): A single validation step in a Contract, defined declaratively
- [Pipeline](reference/Pipeline.html): Binds source and target contracts into a pipeline boundary enforcement unit
- [PipelineResult](reference/PipelineResult.html): Result of a pipeline boundary validation run

#### Contract Import/Export
> Import external schema definitions (JSON Schema, Frictionless Table Schema, and more) into Pointblank validation workflows, or export Pointblank contracts to those formats. Use `import_contract()` as the entry point, `export_contract()` for the reverse, and `register_adapter()` to add support for custom formats.


- [import_contract](reference/import_contract.html): Import a contract/schema from an external format
- [export_contract](reference/export_contract.html): Export a Pointblank validation or contract to an external format
- [list_adapters](reference/list_adapters.html): List all registered adapters with their capabilities
- [register_adapter](reference/register_adapter.html): Register a contract adapter class
- [ContractImport](reference/ContractImport.html): Result of importing an external contract/schema
- [ContractAdapter](reference/ContractAdapter.html): Base class for contract import/export adapters

#### Validation Steps
> Validation steps are sequential validations on the target data. Call `Validate`'s validation methods to build up a validation plan: a collection of steps that provides good validation coverage.


- [Validate.col_vals_gt](reference/Validate.col_vals_gt.html)
- [Validate.col_vals_lt](reference/Validate.col_vals_lt.html)
- [Validate.col_vals_ge](reference/Validate.col_vals_ge.html)
- [Validate.col_vals_le](reference/Validate.col_vals_le.html)
- [Validate.col_vals_eq](reference/Validate.col_vals_eq.html)
- [Validate.col_vals_ne](reference/Validate.col_vals_ne.html)
- [Validate.col_vals_between](reference/Validate.col_vals_between.html)
- [Validate.col_vals_outside](reference/Validate.col_vals_outside.html)
- [Validate.col_vals_in_set](reference/Validate.col_vals_in_set.html)
- [Validate.col_vals_not_in_set](reference/Validate.col_vals_not_in_set.html)
- [Validate.col_vals_increasing](reference/Validate.col_vals_increasing.html)
- [Validate.col_vals_decreasing](reference/Validate.col_vals_decreasing.html)
- [Validate.col_vals_null](reference/Validate.col_vals_null.html)
- [Validate.col_vals_not_null](reference/Validate.col_vals_not_null.html)
- [Validate.col_vals_regex](reference/Validate.col_vals_regex.html)
- [Validate.col_vals_within_spec](reference/Validate.col_vals_within_spec.html)
- [Validate.col_vals_expr](reference/Validate.col_vals_expr.html)
- [Validate.col_exists](reference/Validate.col_exists.html)
- [Validate.col_pct_null](reference/Validate.col_pct_null.html)
- [Validate.rows_distinct](reference/Validate.rows_distinct.html)
- [Validate.rows_complete](reference/Validate.rows_complete.html)
- [Validate.col_schema_match](reference/Validate.col_schema_match.html)
- [Validate.row_count_match](reference/Validate.row_count_match.html)
- [Validate.col_count_match](reference/Validate.col_count_match.html)
- [Validate.data_freshness](reference/Validate.data_freshness.html)
- [Validate.tbl_match](reference/Validate.tbl_match.html)
- [Validate.conjointly](reference/Validate.conjointly.html)
- [Validate.specially](reference/Validate.specially.html)
- [Validate.prompt](reference/Validate.prompt.html)

#### Aggregation Steps
> These validation methods check aggregated column values (sums, averages, standard deviations) against fixed values or column references.


- [Validate.col_sum_gt](reference/Validate.col_sum_gt.html)
- [Validate.col_sum_lt](reference/Validate.col_sum_lt.html)
- [Validate.col_sum_ge](reference/Validate.col_sum_ge.html)
- [Validate.col_sum_le](reference/Validate.col_sum_le.html)
- [Validate.col_sum_eq](reference/Validate.col_sum_eq.html)
- [Validate.col_avg_gt](reference/Validate.col_avg_gt.html)
- [Validate.col_avg_lt](reference/Validate.col_avg_lt.html)
- [Validate.col_avg_ge](reference/Validate.col_avg_ge.html)
- [Validate.col_avg_le](reference/Validate.col_avg_le.html)
- [Validate.col_avg_eq](reference/Validate.col_avg_eq.html)
- [Validate.col_sd_gt](reference/Validate.col_sd_gt.html)
- [Validate.col_sd_lt](reference/Validate.col_sd_lt.html)
- [Validate.col_sd_ge](reference/Validate.col_sd_ge.html)
- [Validate.col_sd_le](reference/Validate.col_sd_le.html)
- [Validate.col_sd_eq](reference/Validate.col_sd_eq.html)

#### Column Selection
> Use the `col()` function along with column selection helpers to flexibly select columns for validation. Combine `col()` with `starts_with()`, `matches()`, etc. for selecting multiple target columns.


- [col](reference/col.html): Helper function for referencing a column in the input table
- [starts_with](reference/starts_with.html): Select columns that start with specified text
- [ends_with](reference/ends_with.html): Select columns that end with specified text
- [contains](reference/contains.html): Select columns that contain specified text
- [matches](reference/matches.html): Select columns that match a specified regular expression pattern
- [everything](reference/everything.html): Select all columns
- [first_n](reference/first_n.html): Select the first `n` columns in the column list
- [last_n](reference/last_n.html): Select the last `n` columns in the column list
- [expr_col](reference/expr_col.html): Create a column expression for use in `conjointly()` validation

#### Segment Groups
> Combine multiple values into a single segment using `seg_*()` helper functions.


- [seg_group](reference/seg_group.html): Group together values for segmentation

#### Interrogation and Reporting
> The validation plan is executed when `interrogate()` is called. After interrogation, view validation reports, extract metrics, or split data based on results.


- [Validate.interrogate](reference/Validate.interrogate.html)
- [Validate.set_tbl](reference/Validate.set_tbl.html)
- [Validate.get_tabular_report](reference/Validate.get_tabular_report.html)
- [Validate.get_step_report](reference/Validate.get_step_report.html)
- [Validate.get_json_report](reference/Validate.get_json_report.html)
- [Validate.get_dataframe_report](reference/Validate.get_dataframe_report.html)
- [Validate.get_sundered_data](reference/Validate.get_sundered_data.html)
- [Validate.get_data_extracts](reference/Validate.get_data_extracts.html)
- [Validate.all_passed](reference/Validate.all_passed.html)
- [Validate.assert_passing](reference/Validate.assert_passing.html)
- [Validate.assert_below_threshold](reference/Validate.assert_below_threshold.html)
- [Validate.above_threshold](reference/Validate.above_threshold.html)
- [Validate.n](reference/Validate.n.html)
- [Validate.n_passed](reference/Validate.n_passed.html)
- [Validate.n_failed](reference/Validate.n_failed.html)
- [Validate.f_passed](reference/Validate.f_passed.html)
- [Validate.f_failed](reference/Validate.f_failed.html)
- [Validate.warning](reference/Validate.warning.html)
- [Validate.error](reference/Validate.error.html)
- [Validate.critical](reference/Validate.critical.html)

#### Inspection and Assistance
> Functions for getting to grips with a new data table. Use `DataScan` for a quick overview, `preview()` for first/last rows, `col_summary_tbl()` for column summaries, and `missing_vals_tbl()` for missing value analysis.


- [DataScan](reference/DataScan.html): Get a summary of a dataset
- [preview](reference/preview.html): Display a table preview that shows some rows from the top, some from the bottom
- [col_summary_tbl](reference/col_summary_tbl.html): Generate a column-level summary table of a dataset
- [missing_vals_tbl](reference/missing_vals_tbl.html): Display a table that shows the missing values in the input table
- [load_dataset](reference/load_dataset.html): Load a dataset hosted in the library as specified table type
- [get_data_path](reference/get_data_path.html): Get the file path to a dataset included with the Pointblank package
- [connect_to_table](reference/connect_to_table.html): Connect to a database table using a connection string
- [print_database_tables](reference/print_database_tables.html): List all tables in a database from a connection string

#### Table Pre-checks
> Helper functions for use with the `active=` parameter of validation methods. These inspect the target table before a step runs and conditionally skip the step when preconditions are not met.


- [has_columns](reference/has_columns.html): Check whether one or more columns exist in a table
- [has_rows](reference/has_rows.html): Check whether a table has a certain number of rows

#### YAML
> Functions for using YAML to orchestrate validation workflows.


- [yaml_interrogate](reference/yaml_interrogate.html): Execute a YAML-based validation workflow
- [validate_yaml](reference/validate_yaml.html): Validate YAML configuration against the expected structure
- [yaml_to_python](reference/yaml_to_python.html): Convert YAML validation configuration to equivalent Python code

#### Utility Functions
> Functions for accessing metadata about the target data and managing configuration.


- [get_column_count](reference/get_column_count.html): Get the number of columns in a table
- [get_row_count](reference/get_row_count.html): Get the number of rows in a table
- [get_action_metadata](reference/get_action_metadata.html): Access step-level metadata when authoring custom actions
- [get_validation_summary](reference/get_validation_summary.html): Access validation summary information when authoring final actions
- [write_file](reference/write_file.html): Write a Validate object to disk as a serialized file
- [read_file](reference/read_file.html): Read a Validate object from disk that was previously saved with `write_file()`
- [ref](reference/ref.html): Reference a column from the reference data for aggregate comparisons

#### Test Data Generation
> Generate synthetic test data based on schema definitions. Use `generate_dataset()` to create data from a Schema object, or `schema_from_tbl()` to infer a generation-ready schema from an existing table (Polars, Pandas, or Ibis/DuckDB).


- [generate_dataset](reference/generate_dataset.html): Generate synthetic test data from a schema
- [schema_from_tbl](reference/schema_from_tbl.html): Create a Schema from an existing table with inferred Field constraints
- [int_field](reference/int_field.html): Create an integer column specification for use in a schema
- [float_field](reference/float_field.html): Create a floating-point column specification for use in a schema
- [string_field](reference/string_field.html): Create a string column specification for use in a schema
- [bool_field](reference/bool_field.html): Create a boolean column specification for use in a schema
- [date_field](reference/date_field.html): Create a date column specification for use in a schema
- [datetime_field](reference/datetime_field.html): Create a datetime column specification for use in a schema
- [time_field](reference/time_field.html): Create a time column specification for use in a schema
- [duration_field](reference/duration_field.html): Create a duration column specification for use in a schema
- [profile_fields](reference/profile_fields.html): Create a dict of string field specifications representing a person profile

#### Prebuilt Actions
> Prebuilt action functions for common notification patterns.


- [send_slack_notification](reference/send_slack_notification.html): Create a Slack notification function using a webhook URL
- [emit_otel](reference/emit_otel.html): Create an OTel export action for use in `FinalActions`

#### Metadata Import/Export
> Import variable-level metadata from external data standards files (CDISC Define-XML, Controlled Terminology, SPSS `.sav`, SAS XPORT, Stata `.dta`, and more) and export metadata to various formats. Use `import_metadata()` as the entry point and `export_metadata()` for the reverse.


- [import_metadata](reference/import_metadata.html): Import metadata from an external standard or file
- [export_metadata](reference/export_metadata.html): Export metadata to an external standard format
- [MetadataImport](reference/MetadataImport.html): Parsed metadata from an external standard
- [MetadataPackage](reference/MetadataPackage.html): A collection of `MetadataImport` objects from a multi-dataset source
- [VariableMetadata](reference/VariableMetadata.html): Metadata for a single variable/column, as imported from an external standard
- [Codelist](reference/Codelist.html): A controlled terminology / value set from an external standard
- [CodelistEntry](reference/CodelistEntry.html): A single entry in a codelist (controlled terminology)
- [MissingValueCode](reference/MissingValueCode.html): A structured missing value definition from an external standard

#### SDTM Validation
> Validate clinical datasets against CDISC SDTM domain templates. Use `validate_sdtm()` to generate a full `Validate` workflow, or `validate_sdtm_structure()` for a quick structural conformance check. Retrieve domain templates with `get_sdtm_domain()` and `list_sdtm_domains()`.


- [validate_sdtm](reference/validate_sdtm.html): Generate a comprehensive SDTM validation workflow for a dataset
- [validate_sdtm_structure](reference/validate_sdtm_structure.html): Validate the structural conformance of a dataset against an SDTM domain template
- [sdtm_to_metadata](reference/sdtm_to_metadata.html): Convert an SDTM domain template to a `MetadataImport` object
- [get_sdtm_domain](reference/get_sdtm_domain.html): Get the SDTM template for a specific domain
- [list_sdtm_domains](reference/list_sdtm_domains.html): List all available SDTM domain codes
- [SDTMDomainTemplate](reference/SDTMDomainTemplate.html): Structural template for an SDTM domain
- [SDTMVariableSpec](reference/SDTMVariableSpec.html): Specification for a single variable in an SDTM domain template

#### ADaM Validation
> Validate analysis datasets against CDISC ADaM templates. Use `validate_adam()` to generate a full `Validate` workflow, or `validate_adam_structure()` for a quick structural conformance check. Retrieve dataset templates with `get_adam_dataset()` and `list_adam_datasets()`.


- [validate_adam](reference/validate_adam.html): Generate a comprehensive ADaM validation workflow for a dataset
- [validate_adam_structure](reference/validate_adam_structure.html): Validate structural conformance of a dataset against an ADaM template
- [adam_to_metadata](reference/adam_to_metadata.html): Convert an ADaM dataset template to a MetadataImport object
- [get_adam_dataset](reference/get_adam_dataset.html): Get the ADaM template for a specific dataset
- [list_adam_datasets](reference/list_adam_datasets.html): List all available ADaM dataset template names
- [ADaMDatasetTemplate](reference/ADaMDatasetTemplate.html): Structural template for an ADaM dataset
- [ADaMVariableSpec](reference/ADaMVariableSpec.html): Specification for a single variable in an ADaM dataset template

#### Integrations
> Classes for integrating Pointblank with external observability and monitoring systems. Use `OTelExporter` to export validation results as OpenTelemetry metrics, traces, and logs.


- [integrations.otel.OTelExporter](reference/integrations.otel.OTelExporter.html)