Get 30x speedups when reading databases with ConnectorX + Arrow + dlt | dlt Docs
Resource type
Web Page
Title
Get 30x speedups when reading databases with ConnectorX + Arrow + dlt | dlt Docs
Abstract
If rust + arrow + duckb is a new data engineering stack, now you can get a feel of it with dlt. We recently added native arrow tables (and panda frames) loading. What it means? You can pass an Arrow table to dlt pipeline.run or pipeline.extract methods, have it normalized, saved to parquet and loaded to your destination.
Date
10/23/23, 12:00 AM
Accessed
11/6/23, 4:46 PM
Language
en
Citation
Get 30x speedups when reading databases with ConnectorX + Arrow + dlt | dlt Docs. (2023, October 23). https://dlthub.com/docs/blog/dlt-arrow-loading
Link to this record