Python Prompts Free Prompt

Data Pipeline Testing Framework

Build a testing framework for data pipelines that catches issues before they reach production
☆ Save
Prompt
You are a data engineering lead and data quality specialist. Create a complete data pipeline testing framework for the following pipeline: [PIPELINE TYPE: ETL/ELT/streaming, DATA SOURCES, CRITICALITY LEVEL]. The framework must cover: 1) Data quality dimensions: completeness, accuracy, consistency, timeliness, and validity, 2) Unit testing for transformations: how to test dbt models or pandas functions in isolation, 3) Integration testing: testing the full pipeline end-to-end with test data, 4) Schema validation: using Great Expectations, Pydantic, or dbt tests for schema enforcement, 5) Volume and statistical anomaly detection: catching unexpected drops or spikes in row counts, 6) Referential integrity testing: validating foreign key relationships across tables, 7) Business rule testing: encoding domain-specific data quality rules, 8) Pipeline regression testing: ensuring changes do not break downstream consumers, 9) Test data management: creating and maintaining representative test datasets, 10) CI integration: how data tests run in the deployment pipeline.

How to use this prompt

1
Click Copy Prompt above to copy to your clipboard.
2
Open ChatGPT, Claude, Gemini, or any AI assistant and paste the prompt.
3
Replace any [PLACEHOLDER] values with your specific details.
4
Run the prompt and refine by asking follow-up questions for better results.