For those that do not feel brave enough to build a weather station from scratch you can buy a 'Watson W-8681' Weather Station and hook up a Raspberry Pi to it to collect all the data.
Jim Easterbrook (ex-BBC engineer) has a really good write up on the process, and also shares his python code:
This is more useful. The mechanics of the IoT is one problem that might need to be solved, but sourcing good sensors is more interesting to read about.
I unfortunately do not have a good site for a weather station, so I just use the NYS Mesonet: http://www.nysmesonet.org/
(They charge $450,000 a year for commercial customers but still don't have a TLS certificate. The weather stations are good, though.)
I got into weather stations just recently. If you don't want to go that far in the DIY way for the hardware I recommend getting [1] + [2] or if you are from EU like me [3].
You can see some of the data here [4]
You can hook this thing up to the internet over wifi and it will send data to https://www.ecowitt.nethttps://www.wunderground.comhttps://weathercloud.net/ and/or post it to your custom endpoint. The gateway can also be queried for data over an API. This thing is quite new on the market but I am very happy with it. You can hook it up to already supported websites or you can use a raspberry pi to collect data yourself (WeeWx and Cumulus MX already support the API).
Good tips, I would like to add that if you already have a weather station you can buy a $10 RTL-SDR like [1] and plug it in to a PI, then use ready made software [2] to extract the weather information. You can hook this data up to any API you would like.
Nice. I love weather station builds. Here is mine. I added a pm air quality sensor, a infrared rain sensor, some light sensors, a CO2 sensor, an accelerometer, and a loudness sensor. There's a spot for my Geiger counter too but that installation is not done yet.
This is cool. You might be interested in the synthetic sensor project - I don't think it ever got to commercial release, but the idea was to try and merge information from lots of different cheap sensors together into "virtual sensors". For example accelerometer + loudness sensor + pressure change could be combined into a "someone shut the door" sensor. Then you could make heirarchies of these virtual signals, e.g. "kettle on virtual sensor" + "morning" + "some IR signature" = "Alice is making breakfast" (rather than Bob). It's quite a neat idea, but as far as I know they never released any code for it.
I should technically solder more of this, as it's mostly together with prototype cables at the moment. Any soldering iron should work, but I am partial to Weller Soldering Stations. I have one from ~10 years ago that has an analog adjustment, though now I see they look to be mostly digital. Their website has them around $300 for some reason but they are on amazon for closer to $100 (which is what I got mine for).
Thanks for sharing this project. I have a lot of small individual sensors tracking mostly temperature and humidity in every room and outside (including one on my bike), and I'm always wanting to build a "complete" weather station, but lack the basis to build it upon. Yours looks cool.
Have you thought about adding some kind of room-temperature spectroscopy along with your Geiger counter? Might be interesting to correlate with spikes in the count rate.
Absolutely. That's part of why I'm delaying. A professor at the University of Michigan is working on a similar project and I want to catch up with her. They're using a cheap(ish) proportional counter to be able to identify all of the isotopes that you might be looking for in a variety of wartime or accident scenarios.
I'm using weewx for my weather station. I pulled the weather station out of a bin at my former place of employment, which is an airmar 200wx, ultrasonic wind sensor that's typically used on boats. The humidity sensor doesn't work, so I wrote a python script, that acts as man in the middle, that reads in the data, and when it gets to the humidity values, it replaces them with numbers from a DHT22 humidity sensor instead.
Living in the PNW with many consecutive months of damp and cool weather, I've found most humidity sensors to be unreliable. Often their operating range is between 20% and 80%, which misses the very dry summers and very wet winters here. After a few weeks, I start getting bad readings, and finally failures.
Lately, I've had better results using a small 5VDC fan within the weather station enclosure, with the hope that dew won't form on the sensor (probably the main culprit). It's a Silicon Labs Si7021 temperature and humidity sensors (I2C). So far so good.
Yes, they do fail. I guess that's evident by the design of the weather station, as the humidity sensor is modular, so it can be replaced.
Unfortunately, I think this design also means it has a low ingress protection value. The unit I ended up with had the humidity sensor replaced and it still wasn't providing numbers. I think it may have spent a year or so out at sea on a buoy.
Thanks for the tip on weewx! Hadn't heard of it, and got my Davis Vantage 2 Pro I just inherited working and posting to weather underground with a raspberry pi + weewx. Real easy
Yeah, it's useful if you want a static website for your weatherstation. But I would quite like the current readings, to actually be current, rather than up to 5 mins old.
I'm currently working on something similar for the a new house I'm building, but for the inside.
I plan to have multiple ESP32 nodes with sensors (luminosity, temperature, pressure, moisture, presence detector), ble scanning (in order to detect who is in the room), and microphones for voice control (picovice porcupine library for stop word detection, then google speech 2 text). For the microphones, I'm still waiting for my mems microphnes to directly include them on the sensor node, or I will use an additional board like the esp32-korvo which works very good (already tested). For audio playback in the rooms, I'm planning on using the up2stream amp devices with ceiling speakers. As for powering it, I'm still unsure on what to use. I first ordered some olimex POE esp32 isolated (so you can power them from both POE and USB at the same time). However the ethernet interface will go up/down in an endless loop after a few minutes of sending data over mqtt. Strangely, the non ISO version works though as it seems, I haven't run it though in a box (higher temperature). So I'm still unsure if I use POE or a 220V->5V converter in each case, or feed them directly with 5V from a central place (which would require more cable length planning)
I did some programming at first, but then quickly switched top https://esphome.io/ which allows you to include your sensors without writing any code. It has support for all kinds of sensors, integrated API, mqtt, webserver, etc... and you only have to configure one yaml file which will then automatically generate the C code and compile/upload your firmware. No need to reinvent the wheel!
It's also easy to include your own components in case you want to do anything custom. The only disadvantage is that it's still based on esp32 idf 3 (in case you are adding components, I would prefer to do it for the new version and not the old one), but the v4 was only released a few weeks ago.
Perhaps this is too off-topic but I was just looking into home automation and perhaps someone might have good advice: I have an old (~25-30 years) radiant ceiling heating system with 15 thermostats that are starting to malfunction and I need to replace. I don't want to break the bank just doing this, but I'd love to have the ability to read temperature and command on/off on each of one.
I'm going through ESPHome right now, but in any case, any recommendations on very low-key but reliable thermostats that are amenable to hacking with them would be super welcome!
If you need to control mains AC loads, I highly recommend something like the Sonoff Wifi switches. They're similar to the wifi "Smart plugs" you can get these days but are aimed at fixed wiring installations and they're safety tested. You can also get versions with built in temp\humidity sensors in the switch, then re-flash the Sonoff firmware with esphome and control all 15 from a single raspberry pi running home assistant. This would allow you to set temperatures for individual rooms over the internet.
1-wire is a great technology for homebrew weather stations and environmental monitoring in general.
Once you get a 1-wire master, you can chain devices on a single cable (e.g. the ubiquitous DS18B20 temperature sensor). Digikey, Adafruit and Sparkfun have plenty of breakout boards and it's pretty easy to find code either bare metal (bit banging) or linux I2C
A word of advice: double-check you're actually getting a BME280. I accidentally bought a BMP280 which lacks the humidity sensor. -_-
So yeah, I have something similar myself, with a React frontend on the official Raspberry Pi display, a BME680 (never got the IAQ stuff to work though) and InfluxDB + Grafana. No remote/outdoor modules yes, because I'm undecided about the technology.
Also, doublecheck the readings from this sensor. There has been plenty of report about the temperature reading being much higher than the actual temperature (in my case, I got 4C higher than ambient). Some people says it has self-heating problem since if you put active airflow the temperature dropped.
Yes, the BME280 is very good at sensing temperature deltas, much less at sensing absolute temperature. A correction factor is most often needed with this device.
I get that people like to play with hardware but for practical purposes, there are public APIs with airport weather station data that anyone can use, published at 5 minute intervals (typically).
Considering the data they output must conform to aircraft avionics parsers that are sometimes decades old, they are unlikely to ever change or go away. Since the weather observations are required for IFR flight, the FAA is quick to repair them when they break, too.
Whatever you can build with hobbyist hardware it won't compare to the airport's equipment.
Lots of reasons why you might want more local though. My local airport is more than 10km away (but there are other monitoring stations). Pilots and controllers assume some uncertainty because even within the airport perimeter there will be temperature and wind variation, but they will be in-calibration which is the main advantage. Controllers often rely on pilots to make reports if conditions differ to what the airport is recording, so it's not always considered the absolute truth.
At home, for example: more accurate irrigation control based on local rainfall, greenhouse control, microclimate variations (which can be significant if you live in a hilly or mountainous region), observatory control for astronomers etc. There are lots of reasons why your local conditions could vary considerably from the airport, but you do need to put some effort into enclosures to avoid biases from solar radiation. You can also add other sensors like buried temperature sensors to monitor the soil, cloud cover, etc.
Microclimate is significant. If you live in a city, you'll likely have urban heating effects. If you live up a hill, you will have significantly different measurements to an airport in a valley. Anyone who's been hiking knows that you absolutely need local observations to assess conditions (eg from a hut). Though this is more for illustration, if you're hiking there are normally special weather services for common routes (Austria has a nice site for this I think).
The Met Office runs an API where you can contribute your own measurements [1]. Sites are rated based on quality of instrumentation - note that the gold standard here could be a calibrated mercury thermometer in a Stephenson screen, nothing special. Calibration is key.
These simple weather stations can also be used indoors as cheap multi room thermometers. You can pair them with air quality, gas and particulate sensors. Also a nice exercise in mesh/multi sensor networking.
Yes, I should have qualified that the airport API thing is much more practical in the US, I suppose, since we spend more on aviation than other countries.
Our government since WW2 has subsidized any city which wants an airport so they can have one.
I just took a look at the Dallas, Texas airspace as an example and there are 14 weather stations within the ring defining the metro area's restricted airspace. It looks like the airspace around London-Heathrow has 3.
Here you'd probably want to use local automatic weather stations (AWS), which are a superset that includes major airports. There are lot more of those around London than just Heathrow. I imagine NOAA has a similar setup.
I'm not sure if the CAA and Met Office share stations though. Airports often have more specific instruments like ceilometers to give measurements of cloud layer height, while AWS might have more cliamtologic things like pyranometers for sunlight hours.
I live in the US and use my local weatherstation to control my basement dehumidifier and decide whether or not to turn the thermostats. The airport is about 10miles away but actually doesn't have all of the data that I need, such as soil temp, leaf moisture, and it's not very local to my specific house which has lots of trees and tends to deviate from the airport that is more exposed. I still use the airport data, but integrated with my own local data to get a better picture of what's going on.
My understanding is that the FAA sites aren't as useful as they could be for weather forecasting. Apparently many sites are too close to the ground, so the temperature data and dewpoint data is unusable for mesoscale analysis. Sometimes the dewpoint is above the air temperature, for example, which is impossible. It doesn't really matter for air traffic, but it does matter for forecasting storms.
If you are interested in the wind direction or the height of the cloud bases, the aviation weather stations are OK. If you are interested in temperature data, you will have to look elsewhere. (Many states have observation sites designed for weather purposes. The Oklahoma Mesonet was the first, but other states have them too. Often these sites have very interesting information, like soil moisture and temperature, and temperatures at varying heights above the ground: http://www.nysmesonet.org/about/sensorshttp://www.mesonet.org/index.php/site/about/instruments)
Don't know where this criticism comes from but temp and dewpoint data absolutely matter for air traffic, because it determines cloud bases. If cloud bases are below the decision height for the instrument approaches into an airport, it's illegal for a commercial operator to depart from there (because they could not reliably return to the field in case of emergency after takeoff).
The standard data output of an FAA weather station is in degrees Celsius, however, there is no accuracy more than that.
One practical example of this hyper local data would be for automatic gardening. In LA, the temperature can range 20 degrees in 10 miles on a given day. It can be cool and cloudy by the beach and hot and sunny by downtown. This makes Dark Sky the only weather app that is somewhat accurate in LA.
If you are gardening in LA, and you knew the hyper local wind conditions in your plot, the light level, and the ambient moisture, you could develop an automatic watering solution for your garden that is truly hands free in all operation and can be customized based on individual plants needs. Many existing auto watering systems use soil sensors that are known to corrode and need replacement, so if you can infer soil moisture content from other non corrosive means that would be much preferred. You'd probably want to use the soil sensors at least initially to get an understanding of the function between your weather data and the particular soil composition of a given plant, but then they wouldn't have to be continually replaced.
After the recent very hot and dry winds from the desert (normally wind comes west) killed off some of my more temperamental plants last week, I'm thinking about some sort of automatic solution like this.
The only exception to that rule I would say is it might be useful to have your own lightning strike sensor if your nearest airport doesn't have one. Lightning strike data is highly local and time sensitive. You can learn a lot about which clouds a small airplane should fly through 'right now' and which ones you should not, just looking at lightning strikes.
Otherwise, your tax dollars are paying for those airport readings, use the data you're footing the bill for already, imo ;).
This seems to be the same hardware that is used by the luftdaten.info [0] project from Germany to track fine dust in the atmosphere. They ship a firmware which is setup to connect to a local WiFi network and transmit your sensor data to a map.
In an easier setup, there is the open-source Ruuvi device (https://ruuvi.com) that works well: it captures temperature, humidity, pressure and delivers the data through bluetooth in an open/documented format, so you can do whatever you like with it after.
I use a few of them with a NUC server and some custom nodejs scripts.
One thing that really improves your experience is storing your data in something that grafana can read. I like graphite, but your mileage may vary.
There are lots of DBs that you can use, including postgres. the flexibility and configurability that you get is outstanding. With the alerts subsystem you can easily attach triggers for bots.
Also having really simple and configurable graphs that look good.
I'm by no means an expert on this, but I had some issues with the temperature reading of the BME280. It seems to be so sensitive that simply having it on top of a Wemos D1 Mini shield PCB cause the temperatures to differ by more than 2 degrees celsius because the heat of the ESP module seems to radiate up to the BME280 (or transfers through the wires/traces). Just having my finger right above it ticked off the temperature reading, so I assumed even indirect sunlight on it or similar things might give wrong readings too.
I resorted to adding a DS18B20 for temperature readings and only using the air pressure and humidity of the BME280. I could design a PCB that moves the BME280 further away from the D1 Mini with tiny traces to avoid heat transfer, but then again I didn't want to risk it and a quick research showed that the DS18B20 seems to be reasonably accurate as well.
The author didn't say so, but I'd assume these sensors must be placed in a small oven of some kind, so as to introduce averaging and hysteresis of temperature.
When I brew my beer, I need to look at the environmental temperature. I can't stick the thermometer in the beer because it'll be contaminated. So I put a thermometer in a glass of water which I place next to the beer vat.
I'd expect you'd have to put that BME280 in a small metal oven, or submerge in a volume of oil or water. It'll screw up the barometer probably.
I'm more curious on how the nice I2c timing diagrams are generated. It doesn't look like the result from wavedrom[1] so I'm wondering if there are more tools like this out there.
Incidentally, I tried doing searches to see if those images are just copied from somewhere else, and ended up on what looks like a news site with what looks like content of the article after multiple passes through google translate [2]. Anyone has an idea why would something like this exist? Somebody playing with AI/ML perhaps?
Is there a network of people installing such weather stations and sharing their info? I thought a crowd-source weather information like that could be very useful.
There's also the Citizen Weather Observer Program [1]. It's more North America-centric (where the data even gets ingested for use by some weather forecast models, after various data quality checks), but anyone from anywhere in the world is also free to share their weather data.
All Netatmo owners can put their local weather station on an open service. You can find it by searching for Netatmo Weathermap and the data is very dense in populated zones. I was able to fairly accurately pinpoint the location of where rain is/was in the town. I'm surprised Netatmo hasn't leveraged this data better.
The UK Met Office runs WOW (Weather Observations Website) which allows users from all over the world to submit their own observations: https://wow.metoffice.gov.uk/
Weatherunderground hater here. There are a large number of stations (Ambient Weather is a big seller in the US) that hook up neatly and conveniently to only supply data to wunderground.
But then you are 100% reliant upon them for their graphs and app displays, which are slow and often broken, and give you little choice in formatting. For example, the app doesn't display a wind history at all, but the web does. But it won't display on a variety of mobile devices very well.
I know they're free (at the advertising level anyway) but I would love to see some competition with better display options and broader browser / device support.
I currently "tee" my transmissions through another server, but only because I have a model of station (no longer made) that made this trivial. Now you have to intercept the RF which is much more of a pain.
https://www.pwsweather.com/ is another alternative to wunderground. Lately weatherunderground has kind of sucked - it has been down for a while, and rather slow -- and I the deprecation of the old API after IBM bought them was a pain to deal with. Still, their API is public, and the data that weatherstations send to it can be intercepted and used in your own system, for example openhab or home assistant.
The Galaxy S4 had a hygrometer, 'ambient' thermometer, and barometer. Most phones have dropped the thermometer (too sensitive to temp fluctuations to make much sense, surface temps not that useful for weather prediction in urban areas, etc) and the hygrometer (waterproofing), but most have kept the barometer.
It's my long passion to 'activate' all the billions of barometers and use them in weather forecasting. There shouldn't really be much need to manufacture so many weather stations when so much can be done using already deployed hardware (smartphones).
I know IBM is doing assimilation of barometer data into their new models from Weather Underground app installs, and I suspect Apple may finally glance at using their iPhone barometers for weather forecasting now that they bought Dark Sky, but I also don't trust either group to make the results and research public and for the greater societal good - I expect Apple and IBM will mostly use their barometer tech for their own internal, private weather forecast systems.
When I come home, my phone connnect to the router and opens the gate and my rpi zero tells me, "Welcome home."
It's been working for a year now every day :D
is a simple python script with openwrt script on router.
I love this! Am super interested in building something like this.
IMO there is even more value if this is built with a central server in mind for each weather station constructed. A network of stations working together can begin to create their own hyperlocal weather network and could even be useful in small-scale, short-term predictions.
There are some startups that do a bigger version of this and for a long time I've been wanting to build a physical weather station network.
Well, I think the most important metric you can measure on the surface is atmospheric pressure. Simple temperature and humidity is generally insufficient to make good predictions - but pressure at the surface, measured across a geography, can tell you about conditions in the upper atmosphere.
I use barometers in phones for this kind of purpose. I think that a minimum of 1 quality reading per 4 sq km would represent sufficiently dense coverage to create an improvement in forecasts. But more is better, since quality can be iffy.
Some papers published by Cliff Mass et al from UW might have more info on density work. I'll try to find a link. [1] [2]
Howdy, seen you posting before. I would be warey of any crowd sourced data. So now we have average american Joe running a weather station that feeds directly into our state of the art GFS model? What if joe forgets to maintain his station? How do you assert quality control?
I just am skeptical about this all. cant trust users phones to provide input for mission critical weather forecast models (GFS). Peoples lives depend on this information!
> So now we have average american Joe running a weather station that feeds directly into our state of the art GFS model?
No way. The data is quality-controlled and bias-corrected before assimilation. Any bad data is thrown out. The papers I linked above do a good job describing how this is done.
> What if joe forgets to maintain his station?
No maintenance required. Individual data points are irrelevant, this idea only works if you can throw out bad data without caring. So throw out any suspicious data.
> How do you assert quality control?
Sorry to keep directing you to the papers above, but a team of really smart people spent years on this topic and they do a better job of describing it than I could. Cliff Mass et al.
> I just am skeptical about this all.
AFAIK the Weather Channel has been providing your forecasts for some years now in Apple and Google products with all of this barometer data from phones and weather stations in backyards. If you didn't notice a strong quality decline since some years ago, then I would say it's fine. You've probably noticed a forecast quality increase during this time I'd guess.
> cant trust users phones to provide input for mission critical weather forecast models (GFS).
Why not though? What is wrong with any of this method? It is all scientifically tested, researched, peer-reviewed, published, analyzed, post-event-analysis, the whole deal.
> Peoples lives depend on this information!
Well yeah, that's why we need more data! We have so little data now that people are dying from preventable cases like a lack of storm warning or good communication about incoming weather events.
This all matters so much. Ignoring billions of deployed, internet-connected barometers is costing lives that could be saved if we used all the data available to us to make accurate weather predictions.
There's some good projects out there for the ESP family that come with drivers for the most common sensors and interfaces to communicate with the most common IOT protocols, and all you have to do is "wire them up" via a web GUI, for example: https://www.letscontrolit.com/wiki/index.php/ESPEasy
I'm a big fan of Openhab along with the Homie 3 mqtt convention. There is nice homie library for the esp8266/32 that makes it very simple to create custom sensors/actuators, that support things like simple wifi based config and OTA firmware updates: https://github.com/homieiot/homie-esp8266
Does anyone know of any projects that allow you to do weather forecasting yourself at home? Of course it's going to be limited without a network of stations, but with measuring atmospheric pressure (and delta) should give you a pretty good indication of the local weather for the next few hours. Maybe you could tie into a receiver for satelite images.
If you collect your own weather data at scale, you can write assimilation routines to get WRF to incorporate your data into the model runs along with NOAA/MADIS/etc-provided data.
I know that's more complicated than you are suggesting but it's worth mentioning.
There are a lot of simple ways to do home weather forecasting. You can write your own Dark Sky clone by downloading/scraping/APIing your local radar station and then write your own wind predictor a few hours into the future based on rain movement or simple statistics from recent local wind data.
good advice. Although running WRF locally, what are you a madman?
I would suggest looking into simple predictions. One of the first ones you learn in school is just basically just repeat what happened today. So if today was sunny, tomorrow sunny. Stupid, simple but decently effective. Except during transition periods between air-masses (when weather is changing).
This is a really cool project that I would perhaps like to try myself one day! A home weather station has been on my list for several years.
Now I will say something negative. It's a nit, but I think it needs to be said.
I've seen a lot of these "In which I write a (perhaps series of) essays on building X using all kinds of cool tech you'd probably like to play with when you could get the same result with less than ten bucks, maybe an hour or two, and just a handful of code, if any"
I'm okay with and completely love "Let's do something cool and learn!" I'm not okay with "I have this problem, so let me by implication teach you how you should architect problems when you run across them" We need to clearly differentiate. You should not, and you should never, start with an important project and begin speculating on what cool tech you could play with.
Playing and learning are great. Yay! Adding structure and complexity where none is needed is an antipattern if you're doing anything of value for yourself or somebody else. This stack might end up being necessary. Absolutely. Make the problem "prove" that it's necessary. You should have a clear and unambiguous process for choosing a stack; do it by necessity. Never even accidentally teach that you do it by "hey, that's cool!" People end up doing that, and somebody ends up having to clean it up.
tl;dr Love fun projects like this. Keep 'em coming. Just please be sure you're clear to the reader that they're learning/fun projects.
* Perfectly fine to add a tech or two to a new project in order to learn-as-you-work, it should just be done with the upmost care and a close eye to how much team bandwidth it's using versus the value being provided and how you'd back it out if you had to.
Hi Daniel. Thanks a lot for the comment. This makes sense. We all should choose right tools for the right job and architect our solutions according to the requirements and restrictions we have.
One of my original goals was to make the financial entry-barrier as low as possible. Another — to provide educational context for covering a full-stack IoT development from firmware to backend to frontend. This means using cheap hardware and open-source software. In fact, I spent around 9$ on everything that was needed for this project.
I wouldn't say that this project is over-engineered with cool tech. I tried to keep the setup as minimal as possible, sticking to industry-standard data exchange protocols and frameworks.
Of course, all of the software part could be written using ANSI C or C++, but from my point of view, it wouldn't make the project simpler, because there are more specialized tools that are better suited for the particular task, be it backend, frontend or firmware (Rust, HTML5+JS and C in case of this project). I have outlined rationalization behind this particular technology stack under the "Choosing the technology stack" heading.
You are right that in a commercial setting, there might be myriad more inputs that constrain the technology choice like team experience, your company's recommended technology stack and etc. Of course, we can't take into consideration all these hypothetical requirements, as they depend on the environment you are working in and the commercial goals of the project.
To make your comment complete, could you share your ideas on how to architect this alternatively to get the same result with less than ten bucks, maybe an hour or two, and just a handful of code, if any? If it is a viable alternative I'll consider adding it into the post as an alternative tech stack, so people reading it will see more alternative options on how to build this project.
Thank you for taking my comment in the spirit it was intended. I'm happy to oblige you.
If your goal is to play and learn about stuff, whatever you're doing is fine.
If your goal is to get basic weather information, buy a used weather station with usb access, plug it up to whatever hobby computer you already have (I have a pi, so I'd use that). Then write a cron to read from the station and write it out to some publicly-available spreadsheet or database. There are a bunch: Google, Airtable, and so on. Most all of these can be written by one POST.
Then you can figure out how you'd like to see the data. Add in a station ID for more fun and a distributed system. I'm seeing Ebay new weather stations as low as 24 bucks or so. That'd be the only cost I know of, and I think with some searching for used gear you could get it under $10.
There's a _huge_ difference between focusing on the problem and focusing on the tech. If you're preparing yourself for a world of corporate coding? Please keep learning about message queues and so forth! You'll need those skills. If, however, you just want to know the weather in various places with your own setup? Do that. Just don't do one of those things while pretending you're doing both. Each project has vastly-different parameters.
To drive this home, if I've decided I want a coder to come to my office and make a cool little way of telling the weather outside? I want the project that just does that. Code for an hour or two, then leave. If you come and start deploying a kubernetes cloud-based auto-scaling megasytem, you're coding for yourself instead of my problem. My only point was that since there is such a huge gap, when you teach people you should be clear about the hidden lessons of what you're teaching.
* Now traditionally programmers throw out a lot of "what if"s here. What if we needed split-second reports? What if we're dealing with an offline situation or a saturated net? All of those are great considerations. Also, all of those were not included in the spec I read back to you. Don't go inventing what-ifs where none existed. It's a good way to build unmaintable monsters. Hopefully you got the point. It was a minor point and not directed at you, just a comment on cool-tech articles in general. (Also, I'd love to demo actually building it, but it would probably be quite less dramatic and interesting than your version, for the reasons I explain above)
Keep up the good writing! Looking forward to reading you again.
If you have a Davis VP2 Weather station, I wrote a blog post a while back reverse-engineering the protocol, and showed how to sniff its wireless transmissions and receive it on something like an IM-ME Pretty Pink Pager. The Davis Outdoor stuff is very good quality for the price and has given me good service for many years.
Why is it relevant? Do you know what they use for forecasts? I work in industry and have built weather applications for 5+ years now. Let me tell you, everyone uses the same source data. Machine learning is not going to help predict weather. If it was that simple, someone would have machine learned the weather already and they would be making zillions of dollars with their accurate AI weather predictions. Do you post climacell just because they say AI and weather?
OP i hate to break it to you. but a weather station wont tell you the right clothes to wear for the day. IT will tell you if it's raining outside currently. Consider this, station says it's raining and cold. You wear lots of clothes, umbrella etc. The rain storm is out of area by 9am and rest of day is hot and sunny.
One thing ClimaCell seems to advertise is that they have access to, "Millions of data points from proprietary sources such as IoT, Drones, Airplanes, Cellular Signals, Sat Com Signals, Cameras, and more, combined with traditional data sources."[1] Another of their pages[2] claims they use signal strength on wireless networks as one of their sensor methods, which is where I assume the "cell" might come from in their name. [3] says, "Instead of relying exclusively on NOAA radar data, as other forecasting services do, ClimaCell also gathers information from closer-to-the-ground sources: cell towers, street cameras, connected vehicles, and internet of things devices such as smart garbage cans situated throughout a city."
I'm not familiar with the industry. Are other providers doing this?
(I assume that the question of whether these sensors are at all useful is another matter entirely.)
Hi. first i'm asking you a question about why you posted climacell, not sure you answered it? Feel free to answer it or not.
To answer your question, if you want to obtain raw sensor data our government (and some volunteers) maintains a network of thousands of weather stations. Many are automated, many are people doing manual obs. If you know how to write code you can programmatically obtain these data.
However you might be confused... climacell does forecasts and you asked about finding station data? They are completely different things. I have no insight into what climacell does but i am a knowledge person in industry. I can tell you that nobody has some groudnbreaking 'AI' machine learning algorithm that is amazingly great at forecasting weather. If such a thing did exist, that person would have zillions of dollars because they can accurately predict weather better then anyone else.
I thought when they said they use existing wireless communication technology, they meant something like Microsoft's distributed signal processing arrays:
Perhaps using 4G and GSM signal quality as an ad-hoc doppler or metric for precipitation. Is their landing page inaccurate? Are they no different from other companies (besides the good at sales and marketing bit)? They seem to have a lot of big names on their customer list.
I also made my own weather display. This one just runs on a RaspberryPi. It can be extended to collect info from other sources (I eventually plan to build some sensors for outside my home), but for now it gets indoor information from my Ecobee, and local weather and forecast from OpenWeatherMaps.
It would be great to have an open weather station platform where you can just plug in new modules to measure new things in an easy straightforward way... An Arduino + modules for the weather world.
I've been wanting to build a fully autonomous (solar panel + cellular network) for a long time...
I've started using https://smartcitizen.me/ for some of my own use. It uses MQTT internally so could probably easily flash it with your own server address to redirect it whenever you wanted.
Very neat -- but I would love to see one of these that doesn't rely on WiFi or Bluetooth, using instead something like a Baofeng UV-5R and sending weather info updates by APRS (amateur radio license required), allowing for really remote observation stations.
The advantage of APRS is that there is a HUGE network of digital repeaters (digipeaters) across the US (and probably world-wide though the US is my main interest). APRS also has several internet cross-over listening points so you could hook into a service like https://aprs.fi to listen for updates hitting the network from your weather stations.
The licensing requirement isn't steep in the US; if memory serves its 35 questions, all multiple choice, and the question pool is published. Readers of Hacker News should be able to pass the technician exam with about a week of light study and that gives you full privileges from 30 MHz and up (APRS runs between 144 and 146 MHz depending on which country you're in).
Built one a couple of month ago with bbc bit ... you have to dig around and I find it most challenging is to understand what those parameters meant. Still have to fix the rain. I thought wind is hard but programming there is no need to do much.
I like that Kirill Dubovikov built a weather station and showed others how. The more people who do this, the better, not just for education and skills, but in terms of gathering weather information that can help forecasters. He wrote, "the Weather app on my iPhone is not very good at making accurate forecasts," and I thought, "he must have been using the wrong app." I hope in addition to building weather stations and inspiring others to do so, he'll also give the AccuWeather app a try. [I work at AccuWeather; I admire the work my colleagues do to make and deliver accurate forecasts.] Even if Kirill prefers his weather station, I hope he'll use good apps to cross-check his station. Kirill can help in other ways. I speak Russian, so I volunteered to help beta-test the upcoming new app in Russian. My whole iPhone is in Russian at the moment. But Kirill is a native speaker.
One issue with hobbyist weather stations is that they aren't particularly useful for use for assimilating into numerical weather models. The pressure is useful as it is relatively hard to get that wrong unless mislabelling sea-level pressure as station pressure.
Other variables can be significantly in error due to the siting of the station: under a tree, not aspirated, in the wake of a building, on a rooftop, etc.
I think wind it particularly hard to get right. However, any station contributing to the CWOP can be looked up on 'weather.gladstonefamily.net' for really nice quality control stats against other local stations. For reference, here is one I randomly picked in the Nashville area that is passing quality control for pressure and temperature, but failing the wind QC:
> Other variables can be significantly in error due to the siting of the station: under a tree, not aspirated, in the wake of a building, on a rooftop, etc.
I would think there is still value there, at a sufficient scale.
Gave it a quick test and I think you should invest a bit more into UX. Default screen shows me only current weather and that just isn't useful. Compare it to default Weather app. Or was I supposed to configure this screen?
Navigation inside the app doesn't feel intuitive at all. Maybe it's more similar to default experience on android? No idea. Map/Hourly/Daily buttons don't work as tabs. Daily view arrives with scrolling up animation, but when you scroll to close it there is a large ad banner (well, understandable, but awkward) and for some reason weather news (why would anyone want it there or at all?). I've noticed "close" button later, but it was in bottom right corner for some reason.
Plastering "RealFeel(r)" everywhere was horrible idea. It is corny, unnecessary and reads worse than "feels like".
Thanks. I think the new app coming out next month has a better UX. As for comparison with the default Weather app, I think the underlying forecast is more accurate. [Note, I work not on the app, but in back-end data. I'm happy to forward feedback to the people who work on the app.]
I had been very disappointed with the local weather forecast last summer, and then there was an interview with one of the meteorologists involved and they said they were quite proud of how accurate their predictions had been so far. I was taken aback, as in my mind their "success rate" had been well below 50%.
Then I got my answer: they focused on the temperature forecast, while since I bike to work what I was interested in was "is it gonna rain today?"
I get that it's difficult to predict when and where rain might occur, but I'd much prefer a rough probability scale to nothing. On the bright side, they have a real-time, radar-based 90 minute rain forecast now, so at least if I check it in the afternoon I can jump on my bike and get home before I get wet.
I think part of the problem is not the accuracy of the forecast, but whether it's useful to you, or not.
As an example, they may forecast that 30% of the areas your city will report rain tomorrow. They may have an high level of confidence in this prediction (and verify it to be correct after the fact). It may still not be very useful to you, since the one outcome is so different to the other.
That said, I agree with your that the predictions should be presented in a more scientific manner. In fact, one irritating pattern is that rain probability and the predicted amount of rain sometimes get combined into a single percentage, which no longer represents neither the probability, nor the amount. It's some kind of weird combination of the two.
Yeah, here they report rain just in mm. They might say 3-7.3mm, or they might say 0-4mm. Now I don't care too much about 2.5mm or 6.1mm, but I definitely care about the chance of more than light rain. For others, people with a gardens etc, the amount of water might matter more.
Just last Thursday they reported zero precipitation for the entire day, not a hint of rain. Yet we got 3 hours of medium-heavy showers in the afternoon... not a very uncommon combination. If I had known there was a 30% chance of showers, I would have packed my rain jacket.
From talking with meteorologists, it seems forecasting actual rain is not very hard if you have enough current data on the state of the atmosphere. What can be difficult is mistiness that is just on the borderline between falling droplets and moisture just hanging in the air. Differences can be extremely local in nature. [I'm not a meteorologist, I program computers, but I'm a pilot so I have a pilot's appreciation of and contact with weather.]
Or even the proportion and impact of ice v. water droplets. The modelling of this in a warming world is making the latest global climate models used in the upcoming IPCC AR6 show a higher equilibrium climate sensitivity than the previous edition [0].
Hi geoff - seems like you are pretty knowledgeable here. I have to disagree with "seems forecasting actual rain is not very hard if you have enough current data on the state of the atmosphere" ... Who told you that? accuweather meteologists?
Rain is not easy predict, especially long in advance. Just taking things with a grain of salt here. If you truly believe that forecasting actual rain is not very hard, please start a new weather forecasting company and give it all you got! However i'll tell you, it's not that easy. Just because a computer model 'runs' does not mean it correctly modeled the atmosphere. All models are imperfect bud and those imperfections get amplified by 10x when trying to simulate earth+atmosphere systems
No, I'm not especially knowledgeable, not more than others here. As a pilot, I have 42 years of experience flying through weather and helping get the word out w/r/t hurricanes, e.g., relaying info via ham radio in the West Indies from relays in Florida before getting hit by the storms, back in the 1970s when that was the way, so I have long interest in and appreciation of weather. But I do also have appreciation for my colleagues and their meteorological expertise. They have something called MinuteCast which in my experience has been very accurate in telling me when precipitation will begin or end (except when it is misty). Regarding my "forecasting actual rain is not very hard" remark, you are right that a "long in advance" forecast is hard, but I was talking about the short term. Specifically, a commenter had wanted to know if it was going to rain before going out to exercise, and for that example, I meant a forecast of precipitation was easy.
I believe accuracy may have dropped due to the decreased number of flights collecting data, but Dark Sky in years past has been very effective at forecasting rain. I rely heavily on it since I typically go most of the summer without a top on my Jeep.
Accuweather? Lol please give me that 45 day daily forecast! All apps use the same fucking forecasts that our government puts out. The problem i think is with people's expectations. They dont understand weather and think it's easy to predict like some other things. They dont understand that it is a dynamic, chaotic, non linear system where repeating the same process 1000x will always yield slightly diff results
We track the snow forecasts closely and my experience is that any forecast more than a week ahead is no better than a coin flip and will usually be changed. We joke about Inaccuweather.
To be fair forecasting snow is hard. I'm just generally against certain companies that 'sell' products that are garbage. If you want helpful weather forecasts go to your local government weather office forecast page. Dont trust some giant global company trying to make money selling ad space on their webpage
I have also suggested that our ability to predict the weather at such scale and with such accuracy is one of the coolest inventions in recent history. Thanks for sharing.
I already have a setup with a Raspberry PI so I’m only looking for a temperature sensor that:
- Is rain proof
- Transmits over BLE or WiFi
- Can be battery powered
The way I solved this was by buying cheap weather stations from AliExpress. Many of them come with multiple temperature/humidity sensors powered by batteries - for as low as 70$ for a station with 4 sensors. Those sensors transmit on 433 MH or 838MHz radio which has excellent range (20m+ through walls) and long (months+) battery life.
Just saw some of the references to RTL-SDR in the thread comments and started down the rabbit hole, and it looks like a fascinating area for hobbyists.
Can you provide some links to temperature sensors you've tried that work well with an RTL-SDR receiver? I'm also interested in setting something up with a Raspberry Pi.
This doesn't actually predict weather, though, correct? It just sits on your roof and tells you whether it's raining? So... the same could be achieved by sticking your hand out of the window?
I like data collection too, I guess I was just confused by the first part that was the author talking about their commute and how they would forget an umbrella.
Jim Easterbrook (ex-BBC engineer) has a really good write up on the process, and also shares his python code:
http://www.jim-easterbrook.me.uk/weather/
I don't know Jim, but I have done the same following his instructions.