I gave up on the idea of RSS on an e-reader because RSS is garbage on the vast majority of websites these days. Most of those that support it will only show you the headline and maybe the first paragraph before including a link to the full article. So you're then still opening the article in a web browser to view it--which then sends you back to the well-known "Experimental web browser on Kindle/Kobo/other e-Ink reader is crap" problem). There were only a handful of sites where I could do meaningful article reading within the RSS reader itself, and one of them (ArsTechnica) required paying a yearly membership fee for the personalized full-article RSS feed.
It's far less convenient, but just running the articles through Pocket makes for a superior reading experience (particularly on the Kobo), since Pocket will extract and display only the readable text, and avoid the nightmare that is trying to display your average webpage on an e-Ink reader browser.
> Most of those that support it will only show you the headline and maybe the first paragraph before including a link to the full article.
If you'd like to turn partial feeds into full-text versions, the project I work on might help. Doesn't work on all feeds, but will work on many: http://ftr.fivefilters.org/
Whoa thanks for posting this, I didn't know about your service. I've recently created a new info capture / database process and this tool will come in handy!
I'm plenty annoyed about RSS becoming steadily more difficult to use. I have to run an extension in Firefox to get an RSS icon in the address bar, which is just sad.
That said, I'm finding that for Substack, at least, RSS works just fine. You get the whole article, no fuss, and it works the way it should, it's just https://example.substack.com/feed.
I'm pleased to see Substack do most things right. I almost wonder if their pitch deck explicitly said "we will zig every time Medium zagged".
A lot of commercial sites do it because of lost ad revenue. Some of them, like Ars Technica, give you full-text RSS feeds, if you subscribe to them. I think that's fair.
There are some non-commercial blogs with no ads, nothing but text on them, but still giving you only the title in the RSS feeds. I suspect it's the default configuration of whatever software they're using for the blogs, and they may not even be aware that RSS is a thing. Maybe I should start writing mails to them to inquire why that is and maybe change their minds on that, since I really can't see a reason for it.
FWIW Most of the feeds I subscribe to are full-text.
Most of those that support it will only show you the headline and maybe the first paragraph
A huge annoyance for me too. It's infuriating to me that marketing people are continually breaking things like RSS. It's unfortunate that there's no way for users to give negative feedback to people who abuse a protocol and render it less useful to everyone, which is worse than not adopting it at all.
I think this is because people mostly read a headline and a conclusion of a story and if all of it is contained in the feed, then they have no reason to go to the site. That means RSS is crippling the Ad revenue and makes the site unsustainable.
Guess it depends on the site cuz I have had pretty much excellent experiences overall using RSS on anything (the worst feeds are tech blogs with hand rolled software that end up messing up their RSS feeds every 3 months leading to full resyncs)
I don,t have any experience with RSS on eInk, but I would rather that my RSS only contains only the title and the "pitch"(very short summary of the article, or the first paragraph). If I'm interested, with a single keyboard shortcut, my RSS reader fetches the article text for me, so I still don't have to visit the site and read the whole article from within the reader. I use InoReader, but I know NewsBlur (and probably others) has the same (fetch article text) functionality.
> I don't know what the folks at Amazon were thinking when they make the browser suck this much.
I've heard it's quite simple: the browser was only ever included in the first place as a way to log into public Wi-Fi networks like coffee shops which have a screen you need to log in through.
As long as the browser is good enough to log you in, Amazon's happy. Which is why, after almost 10 years, I think the browser is still labeled as "experimental". (At least last I checked.)
Especially since they were footing the bill for cell data with the early devices, it's hard to imagine they'd really want you to use it for Web surfing for sure. But the device also isn't very well suited to that.
For years, I used my Kindle + Google Voice as my portable “phone.”
I’d get voicemails (transcribed) and text messages to my Gmail and could text back from there. Gmail even had a lite version that worked quite well on the Kindle’s experimental browser. Fortunately, my needs for making/receiving an actual call on-the-go were nearly non-existent so it worked quite well.
Was really sad when they pulled the plug on the free 3G.
> The ESP32 is a microcontroller that has very little RAM and isn't quite suited to deal with HTML and such. So I had to use a Raspberry Pi as a rendering proxy and transforms the RSS and the text on the page so that the ESP32 can digest.
The Kindle doesn't come with a lot of onboard horsepower, and putting the same class of SoC as a Raspberry Pi in one would likely increase cost and have other tradeoffs.
I'm assuming the Kindle is using an i.MX6 or better NXP SOC, because they are basically the only ones shipping an integrated EPD controller with Linux drivers, which should be more than capable of parsing and rendering HTML. The Remarkable series runs Qt on these chips and is able to render ePub documents just fine.
There are numerous e-ink devices that support browsers, typically running Android, though Kobo is straight-up Linux. Mind that a DIY option would likely be far superior from a privacy / antisurveillance perspective.
I picked up an Onyx BOOX a couple of months ago. The stock browser is a slightly modified (and slightly crippled) Chromium fork. Via F-Droid, I've installed Fennec Fox (mobile Firefox), and other browsers are available.
There are some benefits, and compromises, to e-ink displays. Pixels are cheap, paints are expensive, persistence is free, colour is unavailable (well, on grayscale devices, there are now full-colour e-ink displays at not-unreasonable prices).
My biggest persistent gripes are:
- Scrolling on e-ink really is not how you want to navigate, pagination is much preferred. Neither browsers nor most other apps support paginated navigation.
- The assumption that I'm using a palm-sized full-colour emissive display, rather than a monitor-sized, greyscale, reflective one, means that most "mobile-optimised" apps and websites are highly frustrating.
I've inquired in a few places (including HN) about what app design guidelines / practices exist for e-ink. Apparently there are none. This is disheartening.
There is a CSS media query "update" that would in theory allow sites to switch from scrolling to pagination with CSS on devices that do slower updates like e-ink. Unfortunately no browser seems to support it yet.
> I've inquired in a few places (including HN) about what app design guidelines / practices exist for e-ink. Apparently there are none. This is disheartening.
That probably has a lot to do with the intended audience of e-ink. The most consumer facing application are e-readers, a market that is served by a handful of companies. Most of the remaining applications are commercial.
With respect to guidance, you may want to check out the Mobileread forums. There are a few people who develop software for e-readers there. Notably in the section for Kobo, and likely for other e-readers as well. There are also some open source projects that you can look into. KOReader and Plato come to mind. I also recall a mention of an e-ink guideline in a video by Ralph S Bacon on YouTube, so there is something out there. (Apparently colour e-ink displays should be refreshed at a regular, albeit long, interval in order to maintain image quality.) I realize that most of these source are for palm-sized, or smaller, screens but at least they will understand the physical properties of e-ink.
My own experience, a couple of months using an e-ink device heavily, is that there's a great deal of foregone potential.
Battery life, recharge time, outdoor viewability, overall display quality, and the fact that the devices really are general-purpose / mobile devices (there are Linux, Android, and a few other OS variants out there) means that apps can be tailored to them.
What I like about the BOOX is that it's a pretty good e-reader, and ... somewhat OK Android tablet. That's not faint praise, it's actually "I kinda wish it were a worse tablet so I wouldn't be so distracted by it". Given the present state of mobile device addiction, that could be a fairly broad potential market. Even a small percentage of 4--8 billion is a large number.
Use as a terminal disiplay (using Termux) is surprisingly good. Given that terminals were originally based on teletypes, this isn't entirely surprising.
I'll follow up on your suggestions, thanks.
And can confirm that any e-ink device should be refreshed fairly frequently. The BOOX default is 20 displays, in practice 5--10 would be preferable, though that depends in part on the display mode used (the BOOX has five, from highest quality render to fastest update/paint rate).
I've been using an Onyx Boox Note with inoreader for a few months, and it's not super polished, but it's a great device if you're interested in an e-ink RSS reader!
It runs de-googled Android out of the box, so it can run most apps. Some of my other favorite apps to use on it are it's integrated ebook reader with pen support and integrated machine translators, F-Droid, Firefox, Nextcloud to sync my ebook collection with Calibre, and Materialistic for reading HN.
I've also got Twidere for Twitter, Frost for Facebook, Instagram, Telegram, Messenger and Goodreads on it, but I'm trying to wean myself off of those platforms.
Oh yeah, I definitely wouldn't say that Onyx is a super trustworthy company. Nor is the OS super secure or trustworthy. A fully open source OS would be better, and the bootloader is unlocked, so if I had the time I could try to install a better OS. But right now most of my hacking energy is focused on my getting my Nextcloud system fully setup. I see the actual devices themselves as interchangeable.
I don't know, I often find myself having to click random things to find the feature I'm looking for, even having to fall back to DDG a couple of times because I cannot find what I'm looking for.
I just got a Kobo and went with Plato[1] over KOReader and it's been great for my purposes so far. Fewer features, but it has everything I need and performs even better than KOReader (it's written in Rust by the developer of bspwm/sxhkd and isn't carrying as much legacy baggage as KOReader, which has a longer history) with a very straightforward interface. It also has support for hooks, with a Wallabag integration as the example.
Of the listed issues, only find-in-page has been addressed.
It's still not possible to cross-reference articles by tag within the app (this does work on the Web client), nor are tags readily searchable or editable (again, the Web app does somewhat better).
On e-ink, where pixels are cheap but paints are expensive, Pocket suffers greatly from:
- Not providing a persistent scrollbar to indicate position-within-article.
- Not offering pagination as a default, persistent, fixed-render-for-document option. That is, documents are paginated, the pagination is set from the start of the document, not whatever arbitrary point pagination was "flipped" on.
- Access to document annotation features from within the paginated view.
- Ability to add tags directly from selected document text (with an option to edit).
There are other gripes, but Pocket remains immensely frustrating to use in practice, and the more you use it, the worse it gets.
I had the same reading flow but I gradually got annoyed by how many web pages Pocket wouldn’t parse correctly, then how some pages wouldn’t display correctly on the Kobo. For instance some pages would have the text aligned to the right.
Now I’d like to find something as straightforward as the Firefox integration for Pocket to send any page in PDF to my reMarkable tablet.
I did this, but found I was saving way more articles than I ever had time for. Before long my backlog of stiff to read was overwhelmingly long, so I quit saving articles.
Would LOVE one of these devices, but unfortunately I don't know if I'll get to have one. Really hope to see more devices that are this form factor in the future. I loved my palm pilot way back when, and would love to be able to replace it.
> I'm not making the server code available now, because it is messy and runs on duck-tape now.
I feel like this is an unfortunately common take. "If it's not widely used, or pretty, or innovative, it's not worth sharing." Chances are I've seen more duct tape on production code than whatever simple hacks you've done to make the server work.
If it works and it's not going to leak your personal data or credentials, just share it. Anybody who's judging you by the state of your repos clearly doesn't understand that coding is iterative and happens better in the open, and is probably safe to ignore. Even if you never get a single star or MR, it's now out there to at least be something people can learn from. And really, all you've done is increase the chances that you might learn from somebody else, or help somebody else out.
Author here. I was actually somewhat worried that someone gonna get to my Pi that has some other stuff running on it, because I am absolutely not doing any security and sandboxing on this nodejs server...
But I guess you made a good point, here you go :) Welcome to the wonderful world of random code copy-and-pasted from Stackoverflow!
Thanks for sharing the code mate. May i also suggest adding a token/password if you want to reduce attack surface. Also from a quick glance, this looks like it is vulnerable to ssrf style attack. This type of service is often vulnerable to it given the nature of it is fetching url on user's behalf. I would suggest either isolate it, have a whitelist of domains that you trust or having iptables to deny internal access.
Thanks for the suggestions. I know this implementation has more holes than a swiss cheese, but I will try to plug them as I figure out how to deal with js callbacks :)
And you may very well know this, but keep in mind that if this is on a LAN only and not port forwarded in from the internet, that substantially reduces your exposure; to exploit anything on your server code they'd need to be running behind your firewall. And there are probably juicier targets at that point.
What you built is super cool. If I would like to hack on an m5paper myself what ressources would you recommend that I start with to kick off a small project?
I read the M5paper Factory Test app [1] and it has tons of interesting details.
m5stack has several other example applications in their repo, but I learned a lot from reading their source code in the test app. Their code is quite sane and organized. You just need to be tolerant to their spelling from time to time, I figure English is not their first language (just like me).
We as a community also need to do a better job of embracing unpolished projects and emphasizing a “don’t be bashful” approach. I’ve seen far too many hobbiest projects posted here where the comments have nitpicked them apart in a derogatory, non-constructive way.
> ...a better job of embracing unpolished projects and emphasizing a “don’t be bashful” approach...
Absolutely. One thing I've found is that the tone of HNers varies a lot. As you pointed out, many are very constructive ("have you thought about...") while some are more critical ("you don't have ..." or "the UX is horrible").
If more people could work on doing the former, then I think we can create a more constructive environment for people to do this.
In my opinion, work in progress type projects (or "research projects" should have only constructive feedback, while Finished Products(tm) can take a more critical approach to comments.
As someone who wants to try his code, this is easy to say. But more often than not, hacks and imperfections in the code will be pointed to and laughed at. Companies interviewing him might think less of him, etc. They will get angry with him if he doesn’t fix bugs they find. And do we really expect the community to be appreciative of what he’s done? Open source developers talk about how they maybe make $100 in donations for libraries that millions of people use.
It’s easy to tell someone to share something without understanding there’s usually more harm than good.
I think I write decent code. But I haven't always. And I've at least always tried to put everything I can up on my public GitHub, and even when my employer has been cool with it, just tossing generic non-competitive code (not core to the business) up on GitHub as well. My experience has been the opposite of what you say. I've never had a single person mock me, I've only ever really learned things from comments/issues, gotten value add from PRs, or been encouraged to see that people like/use my code. It's been one of the single greatest sources of learning in my career.
Quite honestly, the employers who are out there that would look at code I've put out there, see a bug, and pass up my resume, are the kind of employers I would almost certainly be just wasting time with if I was to work there. So in that sense it probably only does me good too; I get to spend my time at companies that understand the journey, understand differences of opinion and design, and don't expect perfect code that 100% matches their style guide every time.
The reason I write code is not at all exclusively to make money. And the code I use daily is free and open source by probably a factor of 10:1. So I like to give back where I can. I'm not looking to make a buck, but I'm also obviously gonna put my GitHub on my resume because I'd much rather be matched to employers who can observe my approach and appreciate it enough to bring me to the interview stage.
The only people who care what the code looks like are the people who are jealous they can't do this and want to shit on someone else's project. Honest, no serious coder gives a shit what the code looks like. Only loudmouthed trolls do.
I’ve said this kind of thing before, usually it’s because the code I’m not releasing has a number of dependencies that are pre-installed on my machine. For something like a Node.js server, this isn’t usually a problem since everything is in package.json. But for more complicated stuff, I often have to write an installation guide, which takes time (you really have to test an installation guide on another machine or you _will_ leave out something critical and non-obvious).
This may sound like perfectionism, but if you have incomplete installation instructions, many of the people currently complaining about lack of code will complain about that instead.
> Anybody who's judging you by the state of your repos clearly doesn't understand that coding is iterative and happens better in the open, and is probably safe to ignore
Counterpoint: I applied for a job a few months ago which required a coding sample. I listed a link to a project of mine on GitHub. Rather than commenting on that project, they instead found another repo under the same account -- an open source project I started 6 years ago, but had not returned to since -- and told me they weren't happy with the quality of the code they saw represented there, which killed my application.
I was annoyed. That wasn't a project I had looked at in some time, and I don't feel it represents my best work. It was just a proof of concept that I hoped to later refine. But it cost me an employment opportunity.
I was messing around with the BMW iBus a while back and found a few repos with complete garbage code that was super useful for getting an idea of what the protocol looked like. Of course I didn't end up sharing mine, but ended up selling that car before getting beyond "wow it works!" :(
I had a 2003 530i, which didn't have aux in, nor bluetooth.
My use of the iBus was to emulate the CD changer in the trunk (basically just a heartbeat every 7s or so) which allowed me to use that line-in. I also picked up button presses on the bus in order to respond and control audio playback. Tied it all together by using a raspberry pi as a bluetooth sink and command hub between connected phone and the iBus.
To be fair, that car saw quite a few instances of the other type of "working on the car" too. I have 58 entries in my work log from two years of ownership...
At the same time, there's been plenty of things that I've held off sharing for a week or two, for a chance to clean things up, and this isn't a crazy thing to do.
Which already has inspired me to think about doing something similar only for the whole web. I think I could scale it down to be usable on a smart watch too.
I have one of these M5Paper devices, and it's quite nice! I quickly ran into issues drawing nice UIs, so my app actually runs server-side and sends back a PNG that the M5 simply renders to the screen. Is it the fastest thing in the world? Absolutely not, but it makes a great ambient display that's only refreshed every few minutes.
A while ago I made Morning Digest [1] with a similar use case: to read the news on my eReader, and do so prettily. The end result is a daily epub "newspaper", auto-delivered to my device.
It's essentially a complicated script that I run on a server every 24 hours to fetch news from rss lists, and compile them on a (very long!) epub or pdf, complete with index and good typography. It even gets around a few paywalls thanks to a couple tricks, and the output is complete articles with reasonable/good output quality.
I then have a script on my kobo to fetch the daily digest every morning when I open it. On a kindle you could just have a script email the result to it.
Overall I'm really happy with it, it's the perfect solution to read the news for me.
I built something very similar that formats RSS/Twitter/etc feeds into a pdf that gets automatically pushed to my reMarkable tablet 2x daily. It's so much nicer to use than reading news online!
There are a lot of services (I think Amazon may even maintain a Chrome extension for this) that can take a web page and run it thru a "readability" service and then deliver it to your Amazon Kindle's email address which will add it to your library.
Its a pretty decent way to read a good chunk of the content you find online.
Do you have any solution for RSS to Kindle? I would happily pay to have my RSS feeds delivered straight to my Kindle. In fact I just today paid for https://app.newslettertokindle.com/ for very similar reasons.
Was there a big run of 4.7 inch eink screens recently? Lilygo has a similar set of products: 4.7" screen powered by ESP32 with different battery connectors, touch screen overlay, and a few case options (some of which are basically just repurposed phone bodies). Theirs comes out to ~$50 if you get touch + a case
1. Come across interesting long-form articles on the Internet
2. Click the "Save to Pocket" icon in my browser
(some time passes)
3. Read these articles on my Kobo e-ink reader using the Pocket app.
Could the raspberry pi intermediary approach be used to format the data in a way that's more palatable to the Kindle's built-in browser? If you're going to spoon feed the ESP32, why not spoon feed the Kindle?
Just had the thought that it could be used as a handheld Gemini Protocol reader. Gemini (or Gopher) being lightweight and text-only could fit really well.
I actually prefer this approach. Basically go around websites not serving full articles via RSS by parsing them via Readability or mercury-parser or whatever.
Can't back it up, but I'm pretty sure some online RSS services use the same trick. Makes for a better UX than "click here to read the full story".
So critical. So some parsing is done off the device due to performance constraints. You'll need internet access anyway to pull down the RSS feeds, and a middleman Lambda/CF worker won't add much latency. I think it's a perfectly acceptable tradeoff.
There are available eInk with a DSI interface and Linux drivers? It shouldn't be that hard to build a DIY eInk slim tablet with a PiZero or for some, with a CM4.
It's far less convenient, but just running the articles through Pocket makes for a superior reading experience (particularly on the Kobo), since Pocket will extract and display only the readable text, and avoid the nightmare that is trying to display your average webpage on an e-Ink reader browser.