Try Claude
Vulnerability DatabaseSearch CVEs & track new advisories
AI Tools for Data Engineers
AI tools that help data engineers audit pipeline CVEs, research ETL patterns, diagram data flows, and build reliable lakehouse architectures.
Get started for free
Works in Chat, Cowork and Code
apache-airflow@2.8.1
CVE-2024-25142 (CVSS 8.1) — SSRF via DAG trigger endpoint
apache-spark@3.5.0
No known CVEs
dbt-core@1.7.4
No known CVEs
Recommended action
Upgrade airflow to 2.8.4 before deploying
Pipeline dependency CVE scanning
Check every library in your data stack — Airflow, Spark, dbt, Kafka clients, pandas — for known vulnerabilities before upgrading production pipelines. A critical CVE in a Kafka connector can be invisible to your org's standard vulnerability scanner.
Check for CVEs in: apache-airflow@2.8.1, apache-kafka@3.6.1, apache-spark@3.5.0, dbt-core@1.7.4, pandas@2.1.4.
apache-airflow@2.8.1: CVE-2024-25142 (CVSS 8.1) — SSRF via DAG trigger endpoint. Upgrade to 2.8.4. Others: clean. dbt-core, pandas, Kafka@3.6.1 all pass. Airflow upgrade is urgent if the API is exposed to non-admin users.
ToolRouter search_cves
apache-airflow@2.8.1
CVE-2024-25142 · CVSS 8.1 · SSRF via DAG trigger
apache-kafka@3.6.1
No known CVEs
apache-spark@3.5.0
No known CVEs
dbt-core@1.7.4
No known CVEs
pandas@2.1.4
No known CVEs
Framework and tool documentation lookup
Fetch version-specific docs for dbt, Spark, Airflow, and Kafka without searching through outdated blog posts. Get exact API signatures, configuration options, and migration guides matched to the version you're actually running.
Show me dbt v1.7 docs for incremental models: how to set unique_key for a composite key, the difference between merge and insert_overwrite strategies, and how to handle schema changes with on_schema_change.
dbt v1.7 incremental: unique_key accepts a list for composite keys. merge strategy requires a warehouse-supported MERGE statement — works on Snowflake, BigQuery, Redshift. insert_overwrite is partition-based — requires partition_by config. on_schema_change options: ignore, fail, append_new_columns, sync_all_columns. Full YAML examples included.
ToolRouter fetch_docs
unique_key (composite)
unique_key: ["order_id", "date_day"]
merge strategy
Requires MERGE support · Snowflake, BigQuery, Redshift ✓
insert_overwrite
Partition-based · requires partition_by config
on_schema_change options
ignore | fail | append_new_columns | sync_all_columns
Data architecture and pipeline diagramming
Generate data flow diagrams, ERDs, and pipeline architecture charts for technical specs, data governance docs, and onboarding. Get Mermaid output that renders in GitHub and Confluence instantly.
Generate a data pipeline diagram: PostgreSQL CDC → Debezium → Kafka → Spark Structured Streaming → Delta Lake landing zone → dbt models → Snowflake production warehouse → Looker.
Generated Mermaid flowchart with 8 stages. CDC capture shown with Debezium connector on Postgres. Kafka topic partitioning annotated. Delta Lake with checkpoint path shown. dbt transformation layer shows staging → intermediate → mart pattern. Snowflake target schema labeled.
ToolRouter render_diagram
Source
PostgreSQL · CDC via Debezium connector
Ingest
Kafka topics · partitioned by entity type
Stream
Spark Structured Streaming · enrichment + dedup
Store
Delta Lake landing zone · checkpoint path configured
Transform
dbt · staging → intermediate → mart
Serve
Snowflake prod warehouse → Looker dashboards
Lakehouse table format research
Compare Delta Lake, Apache Iceberg, and Apache Hudi on time-travel capabilities, schema evolution, streaming ingestion, and cloud storage compatibility before choosing the table format for your lakehouse.
Compare Delta Lake, Iceberg, and Hudi for a lakehouse with 10TB/day ingest from Kafka, time-travel queries going back 90 days, and concurrent Spark + Presto reads.
Delta Lake: best Spark integration, Z-ordering for query pruning, limited Presto support without Delta Standalone. Iceberg: cloud-native, excellent Presto/Trino support, more portable across engines. Hudi: best for upsert-heavy CDC patterns but higher operational complexity. Recommend Iceberg for multi-engine concurrency; Delta if your stack is Spark-only.
ToolRouter research
| Apache Iceberg | Delta Lake | Apache Hudi |
|---|
Census and economic data for enrichment pipelines
Pull US Census zip-code level data — population, median income, age distribution — for geospatial enrichment pipelines. Validate your enrichment logic against authoritative government datasets without manual CSV downloads.
Pull zip-code level median household income and population for the top 50 zip codes in Texas for a retail site selection enrichment model.
Retrieved 50 Texas zip codes. Highest median income: 78746 (Austin, West Lake Hills) $186K, 77024 (Houston, Memorial) $178K. Highest population: 77449 (Katy) 122K, 77084 (Houston, Energy Corridor) 95K. Data from 2022 ACS 5-year estimates.
ToolRouter get_census
ZipAreaPopulation
78746Austin · West Lake Hills34,200
77024Houston · Memorial41,800
77449Katy122,000
77084Houston · Energy Corridor95,000
50 zip codes · ACS 2022 5-year estimates
Supply chain risk for open-source data tools
Audit new connectors, Airflow providers, and dbt packages before adding them to production pipelines. Abandoned maintainers and supply chain anomalies in data tooling are particularly dangerous — pipelines run with elevated permissions.
Audit these packages before adding to our Airflow pipeline: apache-airflow-providers-snowflake@5.3, astronomer-cosmos@1.4, great-expectations@0.18.
apache-airflow-providers-snowflake@5.3: maintained by Apache, clean. astronomer-cosmos@1.4: actively maintained by Astronomer, no advisories. great-expectations@0.18: clean, maintained by Great Expectations team. All three are safe to add.
ToolRouter audit_packages
airflow-providers-snowflake@5.3
Maintained by Apache · clean · no advisories
astronomer-cosmos@1.4
Maintained by Astronomer · active · no advisories
great-expectations@0.18
Maintained by GX team · clean
Verdict
All 3 packages safe to add to production
Ready-to-use prompts
Scan data stack for CVEs
Check these data engineering packages for CVEs: apache-airflow@2.9.0, apache-spark@3.5.1, dbt-core@1.8.0, kafka-python@2.0.2, pandas@2.2.0, sqlalchemy@2.0.28. Flag anything CVSS 7+.
dbt incremental model docs
Fetch dbt v1.8 documentation on incremental models. Show unique_key with composite keys, merge vs insert_overwrite vs append strategies, and how to handle late-arriving data with a lookback window.
Data pipeline architecture diagram
Generate a Mermaid data flow diagram: Postgres CDC via Debezium → Kafka topics → Spark Structured Streaming → Delta Lake → dbt staging/intermediate/mart layers → Snowflake → Tableau dashboard.
Iceberg vs Delta vs Hudi
Compare Apache Iceberg, Delta Lake, and Apache Hudi for a lakehouse with: 5TB/day CDC ingest, time-travel 90 days, concurrent Spark and Trino reads, and schema evolution for 200+ columns. Include a recommendation.
Census zip-code enrichment data
Pull 2022 ACS 5-year estimates for zip codes in the Chicago metro area: median household income, total population, median age, and percentage with bachelor's degree or higher.
Airflow provider package audit
Audit these Airflow packages for supply chain risk: apache-airflow-providers-google@10.14, apache-airflow-providers-aws@8.18, astronomer-cosmos@1.5, airflow-dbt@0.4. Check maintainer activity and known advisories.
Kafka Streams vs Flink
Compare Kafka Streams, Apache Flink, and Spark Structured Streaming for real-time enrichment of clickstream events at 500K events/sec with exactly-once semantics and 5-second latency SLA.
Spark partitioning docs
Fetch Apache Spark 3.5 documentation on DataFrame partitioning: repartition vs coalesce, partition pruning with predicate pushdown, and optimal partition size for S3 reads with Parquet files.
Tools to power your best work
Academic ResearchSearch papers, authors, citations
Deep ResearchAI research reports with citations
Library DocsUp-to-date docs and code examples for any library
Diagram GeneratorRender Mermaid, PlantUML & more
Economic DataCensus, FRED, and labor stats
Supply Chain RiskPackage, dependency & exploit risk
165+ tools.
One conversation.
Everything data engineers need from AI, connected to the assistant you already use. No extra apps, no switching tabs.
Pipeline upgrade safety check
Before upgrading any core data tool, check for CVEs in the new version, review breaking changes, and update architecture diagrams.
New data source onboarding
When adding a new data source, research ingestion patterns, validate connector packages, and document the pipeline architecture.
Lakehouse architecture decision
Research table format options, validate the technical approach, and generate a diagram for the RFC before committing.
Frequently Asked Questions
Can Vulnerability Database check Python packages for data engineering CVEs?
Yes. The Vulnerability Database searches by package name and version across the full CVE catalog — it covers PyPI packages like apache-airflow, dbt-core, pandas, and sqlalchemy. Paste the package names and versions from your requirements.txt or Pipfile.lock.
Does Library Docs cover dbt, Airflow, and Spark documentation?
Yes. Library Docs fetches documentation from official sources for all major data engineering tools. Specify the version in your prompt to get version-matched docs — important for tools like dbt and Airflow where APIs change significantly between major versions.
Can Diagram Generator produce Entity-Relationship Diagrams for database schemas?
Yes. Diagram Generator supports ERD syntax — describe your tables and relationships and it outputs a diagram in Mermaid or PlantUML that renders in GitHub, Confluence, and Notion.
What US Census data is available in the Economic Data tool?
Economic Data covers US Census Bureau data including ACS 5-year estimates at zip-code, county, and state levels — population, income, age, education, housing, and commute data. It also covers 800,000+ FRED time series for macro indicators.
Does Deep Research provide technical depth for lakehouse comparisons?
Yes. Deep Research synthesizes official documentation, engineering blog posts from Databricks, Netflix, and Uber, and academic papers into a structured comparison. You get concrete configuration examples and performance benchmark references, not just high-level summaries.
Give your AI superpowers.
Get started for free
Works in Chat, Cowork and Code