Posts tagged with 'gis'
How (and why) to work with Arrow and DuckDB in Rust
My day job involves wrangling a lot of data very fast. I've heard a lot of people raving about several technologies like DuckDB, (Geo)Parquet, and Apache Arrow recently. But despite being an "early adopter," it took me quite a while to figu ... read more →
Quadrupling the Performance of a Data Pipeline
Over the past two weeks, I've been focused on optimizing some data pipelines. I inherited some old ones which seemed especially slow, and I finally hit a limit where an overhaul made sense. The pipelines process and generate data on the order of hund ... read more →
Searching for TIGER Features
Today I had a rather peculiar need to search through features from TIGER matching specfiic attributes. These files are not CSV or JSON, but rather ESRI Shapefiles. Shapefiles are a binary format which have long outlived their welcome according to man ... read more →