Hacker News new | past | comments | ask | show | jobs | submit login
Noise Mapping with KSQL, a Raspberry Pi and a Software-Defined Radio (confluent.io)
115 points by h3throw on Feb 27, 2019 | hide | past | favorite | 30 comments



<JOKE> As the author of dump1090 I'm going to change the license to require that it is always used together with Redis Streams and not with Kafka. </JOKE>


Us hackernewsians have no humor. I'm taking this seriously and forking dump1090 and calling it dump1337


Bug Report: I'm not receiving any aircraft data on 1337MHz...


OP's python script manually parses dump1090's output[0]. Is this the recommended way to get information with this level of detail from dump1090's output into another program? In the past I've used the SBS1 port because I didn't think to run dump1090 as a sub-process. Next time I use dump1090, I'd rather not use a SBS1 parser if I don't have to.

[0] https://github.com/saubury/plane-kafka/blob/master/raspberry...


Both works but perhaps using TCP port 30001 makes more sense in the context of a serious project:

1. Simpler to upgrade to a distributed setup.

2. Format is stable but with enough pre-parsed information, like the coordinates which are hard to extract.

3. The process doing the data collection can avoid to die together with dump1090 in case of crashes or USB errors. You just need a script to restart dump1090, and the ability to reconnect in the other side.

However I don't remember exactly how the SBS1 port data are ready-to-use exactly, and if there is too much work to do compared to just parsing the raw text output of the program.


Do you mean 30002 (raw output over TCP)? I'm trying to get data out of dump1090, not feed it data. If that's the case, I'd still have to write my own decoder to process the raw adsb packets.


I mean 30003, that outputs things like:

MSG,4,,,738065,,,,,,,,420,179,,,0,,0,0,0,0 MSG,3,,,738065,,,,,,,35000,,,34.81609,34.07810,,,0,0,0,0


That's what I've done in the past; I was just wondering if there was a better way.


I think one could open yet another port and output a JSON entry per line with much more information, and just don't care about the old standards.


I'd be curious to see how this compares with a real-time flight track/noise monitoring system ( https://webtrak.emsbk.com/lax4 at LAX for example).


That’s really cool! I wonder if you can see claps of thunder roll across the sensors. Would also be interesting to use this to detect planes that aren’t squitting ADSB.


This is killer. I’ve been wanting to do something similar to figure out why I have Bluetooth and WiFi connectivity issues at random parts throughout the day. I’m thinking there is some kind of interference and would like to prove that.

This post will serve as a model for that. I’ve had an SDR and a few old Pis lying around for years... time to dust them off.


I've wanted to do related things with wifi/bluetooth, just a heads up though that the typical rtl-sdr you find for 20/30$ can't see the 2.4ghz spectrum which both wifi/bluetooth use (typically)


May not need to worry about that, the interference can be in other bands if it's strong enough to blind the other receivers (There's no such thing as a perfect filter). That said I'd expect that it's going to be someone's microwave, or similar device, that's just flooding the spectrum around 2Ghz also.


> The Raspberry Pi is a little underpowered for serious computation, so I need to offload processing to my local Kafka cluster.

Couldn't this be done on the Raspberry Pi with a O(n) search in the OpenFlights data? Probably less sexy, but a simple grep with location and time range should be enough.


That wouldn't show you the real power of KSQL though.


hmm, still seems like odd usage.

given that in the end the query goes for specific time window, wouldn't "regular" sql work fine (vs event processing) ?


Or with an old surplus Core2 laptop.

Using an RPi with an SDR dongle is already pushing its limits, let alone piling on more load. I can't understand why he'd use it other than for buzzword recognition.

Of course the lower-effort solution would have beem simply to run an FR24 playback for his location at 06:00...


Often it is easier to put the RPi, SDR, and antenna in a good place to receive signals then it is to run high quality antenna cable inside. It also helps isolate your main network from lightning strikes.


I know I'm completely missing the point, but how does knowing which plane is upsetting your cat help with the situation? Are you going to report them to the FAA or something?

P.S. Coming up with examples for things in computer science that aren't absurd is hard. I had to think of an example when I was explaining junction tables for many-to-many relationships to someone the other day. After a minute or so of thinking, I just fell back on that old standard of books and authors.


At first I thought this was about mapping noise around the San Carlos airport, identifier "KSQL".


Same. I skimmed the article, and was really confused I saw the author talking about 787s and A380s, because there's no way one of those would fit on the runway at KSQL.


> The Raspberry Pi is a little underpowered for serious computation, so I need to offload processing to my local Kafka cluster.

The RPI includes purpose built hardware for DSP in the VideoCore so, probably not.

Poe's law in action I suppose.

I'm all for big data but this is a case of nuking a fly.


My understanding was that everything on the Pi except for the video processor was open source. Am I mistaken? Or, are you talking of something different? Or, do you know of an open source library or demonstration of use of the VideoCore for DSP?


There's docs and source for some aspects of the VideoCore. I think someone put together an OpenCL implementation, but I don't know how well it works

Edit: That said, I think any sort of OpenCL/GPGPU solution is overkill for this problem. I run Flightaware's dump1090 fork on my Pi and it seems to only use around 30% of one CPU core.


> That said, I think any sort of OpenCL/GPGPU solution is overkill for this problem.

For the cost/benefit ratio? I think you're absolutely right.

But I think it'd be a neat project to learn anyway :)


When I hear the term "ksql" I think of Arthur Whitney's ksql and the early days of kdb, which dates back to the 90's.


Everything is great in this post except ksql license.


The Confluent Community License?


I've been seeing so many Raspberry Pi projects on HN lately. It's great! I just got a Zero W and have a B+ lying around that I don't know what to do with.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: