Introducing Data Standards: One Data Quality Rule for Every Matching Dataset

Introducing Data Standards: One Data Quality Rule for Every Matching Dataset

Introducing Data Standards: One Data Quality Rule for Every Matching Dataset

Fabiana Ferraz

Fabiana Ferraz

Fabiana Ferraz

Technical Writer at Soda

Technical Writer at Soda

Kristof Houben

Kristof Houben

Kristof Houben

UI/UX Designer at Soda

UI/UX Designer at Soda

Table of Contents

Soda Data Standards lets you define a data quality rule once and apply it across every matching dataset. Instead of copying the same checks into separate contracts and keeping those copies in sync, you manage one standard and choose where it applies.

For example, a rule requiring valid, non-missing email addresses can cover customer records, support signups and marketing imports, even when their email columns have different names. One name pattern selects all three, and every address in them must be present and well formed.

Newly onboarded datasets inherit the standard when they match its scope, so coverage grows without someone adding the checks to each new table.

A Data Standards match block with the pattern (?i).*email.* selects the email columns in three customer datasets, then in a new vendors dataset, and skips a new orders dataset with no email column.

This post shows how to write a standard, choose the datasets it covers, and review its results across your organization.

Key Takeaways

  • A data standard is a set of data quality checks you write once. Soda applies it to every dataset its scope selects, including datasets onboarded later.

  • It uses the same Soda contract language as data contracts and matches columns by name pattern or attribute.

  • Each dataset runs it on its own schedule, and the standardโ€™s page shows whether the rule holds everywhere it applies.

What is a data standard in Soda?

A data standard is a reusable set of data quality checks that you write once and deploy across many datasets. It uses the same check language as a data contract, but instead of belonging to one dataset, itโ€™s bound to a scope, which selects every dataset the checks should cover.

Governance and platform teams can use a standard for requirements shared across datasets, such as valid email addresses or required schema checks.

Watch the six-minute demo where we build an email standard on a Postgres data source, from scope to first results.

How is a data standard different from a data contract?

A data contract is a tailored agreement for one dataset. A data standard is one rule that should hold in every dataset it applies to. They run on the same check engine and work together: a dataset can carry its own contract and several standards at the same time.


Data standard
Data contract
Scope
Many datasets, selected by a scope rule
One dataset
Who sets the rule
A governance or platform team, for the whole organization
The datasetโ€™s consumers, with the producing team committing to deliver it
New datasets
Inherit a matching standard automatically
Need a contract of their own
Best for
Rules that hold everywhere: valid emails, non-null IDs, schema checks
Dataset-specific requirements agreed with its consumers

For the producer-consumer agreement behind a contract, see our guide to data contracts.

A script or contract template can distribute the same checks across datasets. Data Standards also manages which datasets receive them, who owns the standard, and how its results are reviewed.

Teams can maintain the shared definition and inspect its coverage without keeping separate copies aligned.

How do you write one rule for every column?

You write the checks as you would in a contract, but replace the columnโ€™s name with a match block that selects every column the checks should apply to. Soda resolves that block against each datasetโ€™s real columns, by name pattern or by an attribute from your glossary or catalog, so one entry covers columns with different names.

Match columns by name

The pattern matches any column whose name contains โ€œemailโ€, in any case. The exclude block removes email_archived and email_opt_in so the address checks do not run on those fields.

Each qualifier gives an expanded check a stable identity, so its results remain traceable over time. Standards created in Soda Cloud fill in qualifiers for you.

columns:
  - match:
      pattern: "(?i).*email.*"
    exclude:
      any_of: [email_archived, email_opt_in]
    checks:
      - missing:
          qualifier: email_missing
          name: "${soda.data-standards.CURRENT_COLUMN} must not be missing"
      - invalid:
          qualifier: email_format
          name: "${soda.data-standards.CURRENT_COLUMN} must have a valid email format"
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Review the matched columns for other naming exceptions before activating the standard.

The check names use ${soda.data-standards.CURRENT_COLUMN} so each result identifies the column being checked.

For contact_email, for example, the missing-value check is named โ€œcontact_email must not be missing.โ€ The same token can reference the matched column in a SQL expression; it resolves only inside a columns: entry.

This regex illustrates an email format rule. Adapt it to your organizationโ€™s requirements and test representative valid and invalid values before applying it broadly.

Match columns by attribute

Column names arenโ€™t always reliable. If a business glossary or catalog already tags your columns, sync those tags to Soda as column attributes through the API and match on them instead.

Matching by attribute applies the same missing-value and format checks to every column tagged term: Email, including columns such as primary_contact whose names donโ€™t contain โ€œemail.โ€

columns:
  - match:
      attributes: { name: term, equals: Email }
    checks:
      - missing:
          qualifier: email_missing
      - invalid:
          qualifier: email_format
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Choosing where a standard applies

A standardโ€™s scope selects the datasets it applies to; the match block selects columns within those datasets. In Soda Cloud, you can scope a standard by data source, schema or dataset name, choose individual datasets, or use dataset attributes such as domain.

For example, select one Postgres data source and a dataset pattern such as customer_data_* to cover its matching customer tables. You can combine several scope-selection rules in one standard.

Data Standards scope rules in Soda Cloud selecting datasets by data source, schema pattern and dataset pattern.

โ†— Scope rules select datasets by data source, schema and dataset name, including wildcard patterns.

Leave the scope empty and the standard applies to every onboarded dataset.

The editor shows how many datasets match the scope and lets you test the standard against one of them. Review the selection and test results before activating it.

Every standard also needs at least one owner, a user or a group. Custom attributes, such as team or domain, help you filter standards and review results for the part of the organization you manage. Once the rule and scope are ready, choose its state:

  • Draft: saved for review. Nothing is linked and nothing runs.

  • Active: linked to every matching dataset and running on schedule.

  • Paused: links are kept but no scans run, so you can resume without rebuilding the setup.

A data standard's status menu in Soda Cloud with Active and Paused.

โ†— An active standard can be paused from its status menu.

When a standard runs

A standard sets no run time of its own: Data Standards runs its checks on each linked datasetโ€™s schedule. That schedule defaults to the data sourceโ€™s settings, and dataset owners can change it or turn scheduled runs off, so governance owns the rule while each dataset owner still decides when it runs.

To verify a dataset immediately, select Verify on its page. This runs its contract and standard checks together by default.

You can also run a datasetโ€™s active standards from a pipeline using the CLI:

soda data-standard verify \
  --dataset datasource/db/schema/customers \
  --data-source ds_config.yml \
  --soda-cloud sc_config.yml \
  --publish

The command returns a result your pipeline can use to stop when a check fails. The --publish option sends the results to Soda Cloud.

How do you read a standardโ€™s results?

A standardโ€™s page shows whether one shared rule holds across every dataset it matches: how many datasets it covers, how many of its checks pass, and which datasets fall short.

For the email standard, that means seeing which datasets contain missing or malformed addresses.

The Data Standards overview lists standards with their owners, matched datasets, check results, compliance and status, so you can choose which rule to inspect.

The Data Standards overview in Soda Cloud listing standards with owner, matched datasets, check results, compliance and status.

Open a standard to review its matched datasets and their results. From there, use the Checks tab to inspect individual check results and the Scans tab to review executions.

A data standard's page in Soda Cloud showing matched datasets, checks, compliance and its scope.

Check that a scan has run before interpreting compliance. In the demo above, the newly activated standard shows zero compliance before its first execution.

A datasetโ€™s Linked standards panel gives the complementary view: the standards applied to that dataset and their checks.

A dataset page in Soda Cloud marking checks inherited from data standards, with the Linked standards panel.

Each check reports Passed, Warned, Failed or Error, as contract checks do. Notification rules determine who is alerted to failures, and failures can become incidents.

If you store failed rows, the records that broke the rule can stay in your own environment in the Diagnostics Warehouse.

For the investigation that follows detection, see how the RCA agent traces failing checks to their root cause.

What to plan before expanding coverage

Wider coverage works when the rule means the same thing in every selected dataset, dataset owners agree on when it runs and who responds, and the names and attributes you match on select what you intend. Check all three before you widen a standardโ€™s scope:

  • Agreement on the rule. Confirm that the requirement fits every selected dataset. Business and data teams still need to agree on valid values, thresholds and exceptions.

  • Schedules and alerts. Agree with dataset owners on when checks run and who responds to failures. If scheduled runs are switched off, the dataset keeps showing its most recent result, so check execution times before using the results in governance reporting.

  • Matching metadata. Scope and column matching use names and attributes, not the values inside rows. Review whether that metadata selects the intended datasets and columns before widening coverage.

Standards cover requirements your team has defined. Record-Level Anomaly Detection (RAD) complements those checks by looking for changes your rules may miss, using recent records and their history.

How to start with Data Standards

Start with one shared requirement, such as valid email addresses. Test it on a few datasets, go through the results with their owners, and widen the scope once the checks and schedules fit. To use Data Standards, you need:

  • Soda Cloud with a Dataset Standard, Team or Enterprise license

  • Soda Runner 2.0.11 or later

  • Data Standards enabled for your organization, and at least one data source with onboarded datasets

  • The Manage Data Standards permission to create or manage standards

Which rule do you copy into every new table? Bring it to a demo. Book my demo โ†’

Frequently Asked Questions

Soda Data Standards lets you define a data quality rule once and apply it across every matching dataset. Instead of copying the same checks into separate contracts and keeping those copies in sync, you manage one standard and choose where it applies.

For example, a rule requiring valid, non-missing email addresses can cover customer records, support signups and marketing imports, even when their email columns have different names. One name pattern selects all three, and every address in them must be present and well formed.

Newly onboarded datasets inherit the standard when they match its scope, so coverage grows without someone adding the checks to each new table.

A Data Standards match block with the pattern (?i).*email.* selects the email columns in three customer datasets, then in a new vendors dataset, and skips a new orders dataset with no email column.

This post shows how to write a standard, choose the datasets it covers, and review its results across your organization.

Key Takeaways

  • A data standard is a set of data quality checks you write once. Soda applies it to every dataset its scope selects, including datasets onboarded later.

  • It uses the same Soda contract language as data contracts and matches columns by name pattern or attribute.

  • Each dataset runs it on its own schedule, and the standardโ€™s page shows whether the rule holds everywhere it applies.

What is a data standard in Soda?

A data standard is a reusable set of data quality checks that you write once and deploy across many datasets. It uses the same check language as a data contract, but instead of belonging to one dataset, itโ€™s bound to a scope, which selects every dataset the checks should cover.

Governance and platform teams can use a standard for requirements shared across datasets, such as valid email addresses or required schema checks.

Watch the six-minute demo where we build an email standard on a Postgres data source, from scope to first results.

How is a data standard different from a data contract?

A data contract is a tailored agreement for one dataset. A data standard is one rule that should hold in every dataset it applies to. They run on the same check engine and work together: a dataset can carry its own contract and several standards at the same time.


Data standard
Data contract
Scope
Many datasets, selected by a scope rule
One dataset
Who sets the rule
A governance or platform team, for the whole organization
The datasetโ€™s consumers, with the producing team committing to deliver it
New datasets
Inherit a matching standard automatically
Need a contract of their own
Best for
Rules that hold everywhere: valid emails, non-null IDs, schema checks
Dataset-specific requirements agreed with its consumers

For the producer-consumer agreement behind a contract, see our guide to data contracts.

A script or contract template can distribute the same checks across datasets. Data Standards also manages which datasets receive them, who owns the standard, and how its results are reviewed.

Teams can maintain the shared definition and inspect its coverage without keeping separate copies aligned.

How do you write one rule for every column?

You write the checks as you would in a contract, but replace the columnโ€™s name with a match block that selects every column the checks should apply to. Soda resolves that block against each datasetโ€™s real columns, by name pattern or by an attribute from your glossary or catalog, so one entry covers columns with different names.

Match columns by name

The pattern matches any column whose name contains โ€œemailโ€, in any case. The exclude block removes email_archived and email_opt_in so the address checks do not run on those fields.

Each qualifier gives an expanded check a stable identity, so its results remain traceable over time. Standards created in Soda Cloud fill in qualifiers for you.

columns:
  - match:
      pattern: "(?i).*email.*"
    exclude:
      any_of: [email_archived, email_opt_in]
    checks:
      - missing:
          qualifier: email_missing
          name: "${soda.data-standards.CURRENT_COLUMN} must not be missing"
      - invalid:
          qualifier: email_format
          name: "${soda.data-standards.CURRENT_COLUMN} must have a valid email format"
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Review the matched columns for other naming exceptions before activating the standard.

The check names use ${soda.data-standards.CURRENT_COLUMN} so each result identifies the column being checked.

For contact_email, for example, the missing-value check is named โ€œcontact_email must not be missing.โ€ The same token can reference the matched column in a SQL expression; it resolves only inside a columns: entry.

This regex illustrates an email format rule. Adapt it to your organizationโ€™s requirements and test representative valid and invalid values before applying it broadly.

Match columns by attribute

Column names arenโ€™t always reliable. If a business glossary or catalog already tags your columns, sync those tags to Soda as column attributes through the API and match on them instead.

Matching by attribute applies the same missing-value and format checks to every column tagged term: Email, including columns such as primary_contact whose names donโ€™t contain โ€œemail.โ€

columns:
  - match:
      attributes: { name: term, equals: Email }
    checks:
      - missing:
          qualifier: email_missing
      - invalid:
          qualifier: email_format
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Choosing where a standard applies

A standardโ€™s scope selects the datasets it applies to; the match block selects columns within those datasets. In Soda Cloud, you can scope a standard by data source, schema or dataset name, choose individual datasets, or use dataset attributes such as domain.

For example, select one Postgres data source and a dataset pattern such as customer_data_* to cover its matching customer tables. You can combine several scope-selection rules in one standard.

Data Standards scope rules in Soda Cloud selecting datasets by data source, schema pattern and dataset pattern.

โ†— Scope rules select datasets by data source, schema and dataset name, including wildcard patterns.

Leave the scope empty and the standard applies to every onboarded dataset.

The editor shows how many datasets match the scope and lets you test the standard against one of them. Review the selection and test results before activating it.

Every standard also needs at least one owner, a user or a group. Custom attributes, such as team or domain, help you filter standards and review results for the part of the organization you manage. Once the rule and scope are ready, choose its state:

  • Draft: saved for review. Nothing is linked and nothing runs.

  • Active: linked to every matching dataset and running on schedule.

  • Paused: links are kept but no scans run, so you can resume without rebuilding the setup.

A data standard's status menu in Soda Cloud with Active and Paused.

โ†— An active standard can be paused from its status menu.

When a standard runs

A standard sets no run time of its own: Data Standards runs its checks on each linked datasetโ€™s schedule. That schedule defaults to the data sourceโ€™s settings, and dataset owners can change it or turn scheduled runs off, so governance owns the rule while each dataset owner still decides when it runs.

To verify a dataset immediately, select Verify on its page. This runs its contract and standard checks together by default.

You can also run a datasetโ€™s active standards from a pipeline using the CLI:

soda data-standard verify \
  --dataset datasource/db/schema/customers \
  --data-source ds_config.yml \
  --soda-cloud sc_config.yml \
  --publish

The command returns a result your pipeline can use to stop when a check fails. The --publish option sends the results to Soda Cloud.

How do you read a standardโ€™s results?

A standardโ€™s page shows whether one shared rule holds across every dataset it matches: how many datasets it covers, how many of its checks pass, and which datasets fall short.

For the email standard, that means seeing which datasets contain missing or malformed addresses.

The Data Standards overview lists standards with their owners, matched datasets, check results, compliance and status, so you can choose which rule to inspect.

The Data Standards overview in Soda Cloud listing standards with owner, matched datasets, check results, compliance and status.

Open a standard to review its matched datasets and their results. From there, use the Checks tab to inspect individual check results and the Scans tab to review executions.

A data standard's page in Soda Cloud showing matched datasets, checks, compliance and its scope.

Check that a scan has run before interpreting compliance. In the demo above, the newly activated standard shows zero compliance before its first execution.

A datasetโ€™s Linked standards panel gives the complementary view: the standards applied to that dataset and their checks.

A dataset page in Soda Cloud marking checks inherited from data standards, with the Linked standards panel.

Each check reports Passed, Warned, Failed or Error, as contract checks do. Notification rules determine who is alerted to failures, and failures can become incidents.

If you store failed rows, the records that broke the rule can stay in your own environment in the Diagnostics Warehouse.

For the investigation that follows detection, see how the RCA agent traces failing checks to their root cause.

What to plan before expanding coverage

Wider coverage works when the rule means the same thing in every selected dataset, dataset owners agree on when it runs and who responds, and the names and attributes you match on select what you intend. Check all three before you widen a standardโ€™s scope:

  • Agreement on the rule. Confirm that the requirement fits every selected dataset. Business and data teams still need to agree on valid values, thresholds and exceptions.

  • Schedules and alerts. Agree with dataset owners on when checks run and who responds to failures. If scheduled runs are switched off, the dataset keeps showing its most recent result, so check execution times before using the results in governance reporting.

  • Matching metadata. Scope and column matching use names and attributes, not the values inside rows. Review whether that metadata selects the intended datasets and columns before widening coverage.

Standards cover requirements your team has defined. Record-Level Anomaly Detection (RAD) complements those checks by looking for changes your rules may miss, using recent records and their history.

How to start with Data Standards

Start with one shared requirement, such as valid email addresses. Test it on a few datasets, go through the results with their owners, and widen the scope once the checks and schedules fit. To use Data Standards, you need:

  • Soda Cloud with a Dataset Standard, Team or Enterprise license

  • Soda Runner 2.0.11 or later

  • Data Standards enabled for your organization, and at least one data source with onboarded datasets

  • The Manage Data Standards permission to create or manage standards

Which rule do you copy into every new table? Bring it to a demo. Book my demo โ†’

Frequently Asked Questions

Soda Data Standards lets you define a data quality rule once and apply it across every matching dataset. Instead of copying the same checks into separate contracts and keeping those copies in sync, you manage one standard and choose where it applies.

For example, a rule requiring valid, non-missing email addresses can cover customer records, support signups and marketing imports, even when their email columns have different names. One name pattern selects all three, and every address in them must be present and well formed.

Newly onboarded datasets inherit the standard when they match its scope, so coverage grows without someone adding the checks to each new table.

A Data Standards match block with the pattern (?i).*email.* selects the email columns in three customer datasets, then in a new vendors dataset, and skips a new orders dataset with no email column.

This post shows how to write a standard, choose the datasets it covers, and review its results across your organization.

Key Takeaways

  • A data standard is a set of data quality checks you write once. Soda applies it to every dataset its scope selects, including datasets onboarded later.

  • It uses the same Soda contract language as data contracts and matches columns by name pattern or attribute.

  • Each dataset runs it on its own schedule, and the standardโ€™s page shows whether the rule holds everywhere it applies.

What is a data standard in Soda?

A data standard is a reusable set of data quality checks that you write once and deploy across many datasets. It uses the same check language as a data contract, but instead of belonging to one dataset, itโ€™s bound to a scope, which selects every dataset the checks should cover.

Governance and platform teams can use a standard for requirements shared across datasets, such as valid email addresses or required schema checks.

Watch the six-minute demo where we build an email standard on a Postgres data source, from scope to first results.

How is a data standard different from a data contract?

A data contract is a tailored agreement for one dataset. A data standard is one rule that should hold in every dataset it applies to. They run on the same check engine and work together: a dataset can carry its own contract and several standards at the same time.


Data standard
Data contract
Scope
Many datasets, selected by a scope rule
One dataset
Who sets the rule
A governance or platform team, for the whole organization
The datasetโ€™s consumers, with the producing team committing to deliver it
New datasets
Inherit a matching standard automatically
Need a contract of their own
Best for
Rules that hold everywhere: valid emails, non-null IDs, schema checks
Dataset-specific requirements agreed with its consumers

For the producer-consumer agreement behind a contract, see our guide to data contracts.

A script or contract template can distribute the same checks across datasets. Data Standards also manages which datasets receive them, who owns the standard, and how its results are reviewed.

Teams can maintain the shared definition and inspect its coverage without keeping separate copies aligned.

How do you write one rule for every column?

You write the checks as you would in a contract, but replace the columnโ€™s name with a match block that selects every column the checks should apply to. Soda resolves that block against each datasetโ€™s real columns, by name pattern or by an attribute from your glossary or catalog, so one entry covers columns with different names.

Match columns by name

The pattern matches any column whose name contains โ€œemailโ€, in any case. The exclude block removes email_archived and email_opt_in so the address checks do not run on those fields.

Each qualifier gives an expanded check a stable identity, so its results remain traceable over time. Standards created in Soda Cloud fill in qualifiers for you.

columns:
  - match:
      pattern: "(?i).*email.*"
    exclude:
      any_of: [email_archived, email_opt_in]
    checks:
      - missing:
          qualifier: email_missing
          name: "${soda.data-standards.CURRENT_COLUMN} must not be missing"
      - invalid:
          qualifier: email_format
          name: "${soda.data-standards.CURRENT_COLUMN} must have a valid email format"
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Review the matched columns for other naming exceptions before activating the standard.

The check names use ${soda.data-standards.CURRENT_COLUMN} so each result identifies the column being checked.

For contact_email, for example, the missing-value check is named โ€œcontact_email must not be missing.โ€ The same token can reference the matched column in a SQL expression; it resolves only inside a columns: entry.

This regex illustrates an email format rule. Adapt it to your organizationโ€™s requirements and test representative valid and invalid values before applying it broadly.

Match columns by attribute

Column names arenโ€™t always reliable. If a business glossary or catalog already tags your columns, sync those tags to Soda as column attributes through the API and match on them instead.

Matching by attribute applies the same missing-value and format checks to every column tagged term: Email, including columns such as primary_contact whose names donโ€™t contain โ€œemail.โ€

columns:
  - match:
      attributes: { name: term, equals: Email }
    checks:
      - missing:
          qualifier: email_missing
      - invalid:
          qualifier: email_format
          valid_format:
            name: Email format
            regex: '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$'

Choosing where a standard applies

A standardโ€™s scope selects the datasets it applies to; the match block selects columns within those datasets. In Soda Cloud, you can scope a standard by data source, schema or dataset name, choose individual datasets, or use dataset attributes such as domain.

For example, select one Postgres data source and a dataset pattern such as customer_data_* to cover its matching customer tables. You can combine several scope-selection rules in one standard.

Data Standards scope rules in Soda Cloud selecting datasets by data source, schema pattern and dataset pattern.

โ†— Scope rules select datasets by data source, schema and dataset name, including wildcard patterns.

Leave the scope empty and the standard applies to every onboarded dataset.

The editor shows how many datasets match the scope and lets you test the standard against one of them. Review the selection and test results before activating it.

Every standard also needs at least one owner, a user or a group. Custom attributes, such as team or domain, help you filter standards and review results for the part of the organization you manage. Once the rule and scope are ready, choose its state:

  • Draft: saved for review. Nothing is linked and nothing runs.

  • Active: linked to every matching dataset and running on schedule.

  • Paused: links are kept but no scans run, so you can resume without rebuilding the setup.

A data standard's status menu in Soda Cloud with Active and Paused.

โ†— An active standard can be paused from its status menu.

When a standard runs

A standard sets no run time of its own: Data Standards runs its checks on each linked datasetโ€™s schedule. That schedule defaults to the data sourceโ€™s settings, and dataset owners can change it or turn scheduled runs off, so governance owns the rule while each dataset owner still decides when it runs.

To verify a dataset immediately, select Verify on its page. This runs its contract and standard checks together by default.

You can also run a datasetโ€™s active standards from a pipeline using the CLI:

soda data-standard verify \
  --dataset datasource/db/schema/customers \
  --data-source ds_config.yml \
  --soda-cloud sc_config.yml \
  --publish

The command returns a result your pipeline can use to stop when a check fails. The --publish option sends the results to Soda Cloud.

How do you read a standardโ€™s results?

A standardโ€™s page shows whether one shared rule holds across every dataset it matches: how many datasets it covers, how many of its checks pass, and which datasets fall short.

For the email standard, that means seeing which datasets contain missing or malformed addresses.

The Data Standards overview lists standards with their owners, matched datasets, check results, compliance and status, so you can choose which rule to inspect.

The Data Standards overview in Soda Cloud listing standards with owner, matched datasets, check results, compliance and status.

Open a standard to review its matched datasets and their results. From there, use the Checks tab to inspect individual check results and the Scans tab to review executions.

A data standard's page in Soda Cloud showing matched datasets, checks, compliance and its scope.

Check that a scan has run before interpreting compliance. In the demo above, the newly activated standard shows zero compliance before its first execution.

A datasetโ€™s Linked standards panel gives the complementary view: the standards applied to that dataset and their checks.

A dataset page in Soda Cloud marking checks inherited from data standards, with the Linked standards panel.

Each check reports Passed, Warned, Failed or Error, as contract checks do. Notification rules determine who is alerted to failures, and failures can become incidents.

If you store failed rows, the records that broke the rule can stay in your own environment in the Diagnostics Warehouse.

For the investigation that follows detection, see how the RCA agent traces failing checks to their root cause.

What to plan before expanding coverage

Wider coverage works when the rule means the same thing in every selected dataset, dataset owners agree on when it runs and who responds, and the names and attributes you match on select what you intend. Check all three before you widen a standardโ€™s scope:

  • Agreement on the rule. Confirm that the requirement fits every selected dataset. Business and data teams still need to agree on valid values, thresholds and exceptions.

  • Schedules and alerts. Agree with dataset owners on when checks run and who responds to failures. If scheduled runs are switched off, the dataset keeps showing its most recent result, so check execution times before using the results in governance reporting.

  • Matching metadata. Scope and column matching use names and attributes, not the values inside rows. Review whether that metadata selects the intended datasets and columns before widening coverage.

Standards cover requirements your team has defined. Record-Level Anomaly Detection (RAD) complements those checks by looking for changes your rules may miss, using recent records and their history.

How to start with Data Standards

Start with one shared requirement, such as valid email addresses. Test it on a few datasets, go through the results with their owners, and widen the scope once the checks and schedules fit. To use Data Standards, you need:

  • Soda Cloud with a Dataset Standard, Team or Enterprise license

  • Soda Runner 2.0.11 or later

  • Data Standards enabled for your organization, and at least one data source with onboarded datasets

  • The Manage Data Standards permission to create or manage standards

Which rule do you copy into every new table? Bring it to a demo. Book my demo โ†’

Frequently Asked Questions

Do data standards replace data contracts?

No. A data contract is a tailored agreement for one dataset, and a data standard is one rule applied across many. They use the same Soda contract language and check engine, and a dataset can carry its contract and several standards at once. Verify on a dataset's page runs both by default.

Do columns need the same name for a standard to apply?

No. A match block selects columns by regular expression, by an explicit list of names, or by column attribute, and Soda resolves it against each dataset's real columns. Matching on attributes synced from a glossary or catalog covers columns whose names follow no pattern.

What happens when a new dataset is onboarded?

A new dataset is linked automatically to every active standard whose scope it matches, because Soda evaluates scopes continuously. It then runs the standard's checks on its own schedule, inherited from its data source's default. Refresh scope re-evaluates a standard's scope on demand.

Can I run a data standard in CI?

Yes, one dataset at a time. soda data-standard verify, given the dataset, data source and Soda Cloud configuration, runs every active standard that applies to that dataset. It returns the exit codes soda contract verify uses, so a pipeline can stop on a failed check.

Trusted by the worldโ€™s leading enterprises

Real stories from companies using Soda to keep their data reliable, accurate, and ready for action.

At the end of the day, we donโ€™t want to be in there managing the checks, updating the checks, adding the checks. We just want to go and observe whatโ€™s happening, and thatโ€™s what Soda is enabling right now.

Sid Srivastava

Director of Data Governance, Quality and MLOps

Investing in data quality is key for cross-functional teams to make accurate, complete decisions with fewer risks and greater returns, using initiatives such as product thinking, data governance, and self-service platforms.

Mario Konschake

Director of Product-Data Platform

Soda has integrated seamlessly into our technology stack and given us the confidence to find, analyze, implement, and resolve data issues through a simple self-serve capability.

Sutaraj Dutta

Data Engineering Manager

Our goal was to deliver high-quality datasets in near real-time, ensuring dashboards reflect live data as it flows in. But beyond solving technical challenges, we wanted to spark a cultural shift - empowering the entire organization to make decisions grounded in accurate, timely data.

Gu Xie

Head of Data Engineering

4.4 of 5

Your data has problems.
Now they fix themselves.

Automated data quality, remediation, and management.

One platform, agents that do the work, you approve.

Trusted by

Trusted by the worldโ€™s leading enterprises

Real stories from companies using Soda to keep their data reliable, accurate, and ready for action.

At the end of the day, we donโ€™t want to be in there managing the checks, updating the checks, adding the checks. We just want to go and observe whatโ€™s happening, and thatโ€™s what Soda is enabling right now.

Sid Srivastava

Director of Data Governance, Quality and MLOps

Investing in data quality is key for cross-functional teams to make accurate, complete decisions with fewer risks and greater returns, using initiatives such as product thinking, data governance, and self-service platforms.

Mario Konschake

Director of Product-Data Platform

Soda has integrated seamlessly into our technology stack and given us the confidence to find, analyze, implement, and resolve data issues through a simple self-serve capability.

Sutaraj Dutta

Data Engineering Manager

Our goal was to deliver high-quality datasets in near real-time, ensuring dashboards reflect live data as it flows in. But beyond solving technical challenges, we wanted to spark a cultural shift - empowering the entire organization to make decisions grounded in accurate, timely data.

Gu Xie

Head of Data Engineering

4.4 of 5

Your data has problems.
Now they fix themselves.

Automated data quality, remediation, and management.

One platform, agents that do the work, you approve.

Trusted by

Trusted by the worldโ€™s leading enterprises

Real stories from companies using Soda to keep their data reliable, accurate, and ready for action.

At the end of the day, we donโ€™t want to be in there managing the checks, updating the checks, adding the checks. We just want to go and observe whatโ€™s happening, and thatโ€™s what Soda is enabling right now.

Sid Srivastava

Director of Data Governance, Quality and MLOps

Investing in data quality is key for cross-functional teams to make accurate, complete decisions with fewer risks and greater returns, using initiatives such as product thinking, data governance, and self-service platforms.

Mario Konschake

Director of Product-Data Platform

Soda has integrated seamlessly into our technology stack and given us the confidence to find, analyze, implement, and resolve data issues through a simple self-serve capability.

Sutaraj Dutta

Data Engineering Manager

Our goal was to deliver high-quality datasets in near real-time, ensuring dashboards reflect live data as it flows in. But beyond solving technical challenges, we wanted to spark a cultural shift - empowering the entire organization to make decisions grounded in accurate, timely data.

Gu Xie

Head of Data Engineering

4.4 of 5

Your data has problems.
Now they fix themselves.

Automated data quality, remediation, and management.

One platform, agents that do the work, you approve.

Trusted by