helix-ir

helix-ir is a standalone open-source Python library that reads nested, semi-structured data from any source and outputs correctly-typed relational schemas, executable DDL, and a composable transformation pipeline.

Getting Started

• Quick Start
  • Installation
  • Hello World: Infer → Normalize → DDL
  • Transformation Pipeline
• Concepts
  • The Universal Intermediate Representation
  • The Type System
  • The Type Lattice
  • The Schema
  • Normalization Strategies

API Reference

• helix_ir.types
  • HelixType
  • join()
  • meet()
  • subsumes()
  • helix_type_to_arrow()
  • arrow_to_helix_type()
• helix_ir.schema
  • Schema
  • Path
  • PathSegment
  • helix_type_to_json()
  • helix_type_from_json()
• helix_ir.infer
  • infer()
• helix_ir.normalize
  • normalize()
  • NormalizationPlan
  • ForeignKey
• helix_ir.transform
  • Table
  • col()
  • lit()
  • star()
  • sum_()
  • avg_()
  • min_()
  • max_()
  • count_()
  • count_distinct_()
  • coalesce()
  • concat()
  • date_trunc()
  • length()
• helix_ir.pii
  • detect_pii()
  • detect_pii_from_field_name()
• helix_ir.diff
  • diff()
  • SchemaDiff
  • SchemaChange
• helix_ir.ddl
  • compile_ddl()
  • compile_migration()
  • DDLOptions
  • DDLScript
  • get_dialect()
• helix_ir.lineage
  • Lineage
  • LineageEdge
• helix_ir.sources
  • Source
  • JSONSource
  • ParquetSource
  • RestSource
• helix_ir.test
  • generate_tests()
  • run_test()
  • Test
  • TestResult
• helix_ir.cli
  • cmd_infer()
  • cmd_ddl()
  • cmd_normalize()
  • cmd_diff()
  • cmd_lineage()
  • cmd_test()

Indices and tables

• Index

• Module Index

• Search Page