Lakehouse
One store for everything.
A lakehouse keeps raw files and query-ready tables in the same place, so there is one copy of the truth instead of two.
Older setups split data in two: a cheap data lake for raw files, and a separate warehouse for clean tables. Keeping the two in step is a constant chore.
TransitAI Core uses Delta Lake, an open table format that adds database-style guarantees to files in cloud storage. The tables are organized into three layers.
- Table format
- Delta Lake
- Safe writes
- A change applies fully or not at all (ACID transactions)
- History
- Earlier versions of a table can still be read (time travel)
- Shape checks
- Writes that do not match the table are rejected (schema enforcement)
- Cloud
- AWS