On: PgPDF: Pdf Type and Functions for Postgres

[nathanwallace]

Readers may also enjoy Steampipe [1], an open source tool to live query 140+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc with other tables. (Disclaimer - I'm a lead on the project.)

1 - https://github.com/turbot/steampipe

[nlm → nathanwallace]

That's a great piece of software indeed. I use it to join and analyze data from multiple APIs with materialized view, as most data analysis software only connects to databases, not APIs.

It's also pretty easy to write custom plugins once you understand how it's done.

[nathanwallace → nlm]

Thanks for leaning into Steampipe!

[oulipo → nathanwallace]

Interesting! Can you tell me the typical use-case for this? Is it more for dev-ops stuff? Like to query all your servers to do some dashboards of your infra? Or can that be used also for ML / datascience etc?

[nathanwallace → oulipo]

Steampipe is most commonly used for DevSecOps. It works with any BI tool for dashboards (it's just Postgres), but is really great when paired with Powerpipe [1] for security benchmarks, visualizing infrastructure and much more [2]. It can definitely be used for a wide range of data scenarios, e.g. there is an OpenAI plugin [3].

1 - https://github.com/turbot/powerpipe 2 - https://hub.powerpipe.io 2 - https://github.com/turbot/steampipe-plugin-openai

[wvh → nathanwallace]

We've been using Steampipe every once in a while and it works as advertised, though perhaps SQL isn't always the abstraction of choice mentally. Congrats on hitting v1!

[nathanwallace → wvh]

thank you! We believe that SQL makes the simple stuff easy and the hard stuff doable :). Hopefully our repos & hubs have good enough examples to get you started most of the time!

[pestaa → nathanwallace]

This is super cool.

Last time I searched only https://fdw.dev came up (from Supabase).

[mind-blight → nathanwallace]

This is super cool! Do you have any integrations with duckdb? It seems like these two could work really well together

[nathanwallace → mind-blight]

Beyond the Steampipe CLI (which has an embedded Postgres), you can use the steampipe plugins as native Postgres FDWs [1], SQLite extensions [2] or command line export tools [3]. We're definitely interested in bringing the plugins to duckdb as well, but haven't had a chance to focus there yet!

1 - https://steampipe.io/blog/2023-12-postgres-extensions 2 - https://steampipe.io/blog/2023-12-sqlite-extensions 3 - https://steampipe.io/blog/2023-12-steampipe-export

[nkmnz → nathanwallace]

the plugins link from your github repo goes to 404: https://hub.powerpipe.io/plugins :)

[nathanwallace → nkmnz]

doh! thanks for the heads up :-(. Fixed to be https://hub.steampipe.io/plugins

[djbusby → nathanwallace]

I think you mean disclosure.

Why would you disclaim something cool?

[bbor → djbusby]

Ooo love this rabbit hole. I’ve always used “disclaimer” as “warning”, and would definitely use it for a conflict of interest, but technically the legal field uses it specifically for asserting a lack of liability (ish).

Certainly interesting, but I think I’m gonna stubbornly stand my descriptivist ground here. We conquered “literally”, and “sneak peak” is soon to fall — I’ll add this to the list!

[nathanwallace → djbusby]

Fair point. I guess that I'm disclosing that I lead the project and disclaiming the self-interest in posting? :)

[adhamsalama → nathanwallace]

Looks pretty cool. Will try it out.