Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Linkwarden – An open source collaborative bookmark manager (linkwarden.app)
280 points by daniel31x13 on July 31, 2023 | hide | past | favorite | 88 comments
Hey there HN! Meet Linkwarden, a fully self-hostable, open-source collaborative bookmark manager to collect, organize and archive webpages.

Please also visit/star our GitHub repo [1].

Linkwarden was built using TypeScript and NextJS, backed by a PostgreSQL database for the lighter-weight data. The rest of the data can be chosen either to be stored on the filesystem, or stored on the cloud on Digital Ocean Space/AWS S3, the reason for the cloud storage solution was for the Cloud offering [2], we realized that the preserved webpages (archives) take up space pretty quickly and S3 was much more efficient for this task. On the front-end we used TailwindCSS for styling and Zustand for state management.

You could either use our Cloud offering (with 14-day free trial) to directly support this project and experience Linkwarden, or you could self-host it on your own machine and have maximum flexibility.

Feel free if you had any questions, we'll do our best to answer it.

[1]: https://github.com/linkwarden/linkwarden

[2]: https://cloud.linkwarden.app/register - Hosted in Digital Ocean's datacenter located here in Toronto, ON.




This looks really nice, great work. I'll definitely give it a try.

Have you considered a free tier where you could monetize it maybe via sponsorships/ads with the goal to have a social aspect?

I'm a huge fan of Githubs social trending/explore/lists/topics section for finding new tools for specific things that I work on, rust, go, aws, etc. for myself and my teams. Also things like dev.to, daily.dev, etc but they're not really as useful as I thought they'd be. You can see an example of the Lists I've created here https://github.com/mikejk8s?tab=stars - I wind up putting these lists into a team notion doc right now.

There's those "Awesome-XXYZ" lists but I don't think they're the best way to do this at all. They also wind up very out of date. My Github lists aren't collaborative, I can't give people a way to contribute to them and as far as I know they're not something you can search globally to find if someone has some interesting lists.

It's quite a bit different than what you're doing here but what I've been hoping to find was some sort of technology Looking Glass/aggregator where I could click a topic/Collection, say Rust, and see rss feeds, blogs, curated and very well organized bookmarks, hashtags of other related lists, etc in a collaborative manner with lots of contributors.

I was sort-of beginning to do this via a published notion domain and treating it like a wiki.. https://mrj84.notion.site/Go-Wiki-c637ff57e00046bfbe22fb2562... - that's the closest I've been able to brain storm as something remotely near what I'm aiming for.

Sorry for the long post, maybe it'll give you some ideas or maybe someone has some ideas for me.


Amazing. Collaborative bookmark management needed to happen.

Imagine deciding you want to learn a new skill and not having to start from square one learning the lay of the land around your newly chosen subculture but rather it's already been curated for you and you can hit the ground running with great links to various assets and communities like software tools, forums, discords and video tutorials. I love the archiving functionality too.

Your website looks great and all the information I needed was right there. Concise, informative and neat.

I think the real value here is a tool that will allow people of all kinds to aggregate relevant content and help onboard people to the communities they know, live and love.

I feel like a freemium model would allow this tool to reach a massive scale as I see awesome utility in it but I don't expect normal people would self host or pay for it even if it cost 10 cents. It's just too much inertia.

I feel so dirty telling you how to charge for your own product.

Pagerank and SEO killed search. What you have here sounds a hell of a lot better.

Awesome work mate! I'm aways up for a chat.


I'll definitely give it a short this weekend. Are there any plans to support different authentication methods? Like LDAP, OAuth2 etc?

I'm using linkding at the moment https://github.com/sissbruecker/linkding which also has a browser addon, the only missing thing is some form central user auth but we're using it as it is.


Linkding does support header auth if your provider supports that (I run authelia backed by ldap).


Thanks for pointing that out. I'm already using authelia backed by ldap, it'll be an easy setup. I might've to figure how to merge the existing users and their data though.


Currently the only authentication methods are using plain username/password as default.

And if the extra environment variables are set properly, you could hook it up using the email provider, taking care of the confirmation emails and one time links.


If its on nextjs I've a feeling there are auth providers kicking about to implement sso at least.


Absolutely, the authentication is being handled by next-auth so there are lots of providers that can be added in the future.


This looks really neat! Can you share more about the project? Such as:

1. What is the driving vision behind this project? For example is this just scratching a personal itch with hopes it helps others, or is the hope to expand this into a product or company in the future?

2. Is the goal to monetize somehow in the future? If so, what sort of monetization strategies are being considered? For example, "open core", "paid hosting" (what happens to self-hosted?)


Great question, Linkwarden was initially a personal project but then we decided to scale it up into a fully fledged product. Regarding monetization, we already included the paid hosting plan for the users who don’t want to self-host, but the self-hosted option will remain free forever and will always be supported alongside the paid hosting.


To save anyone else the clicks, the pricing is $4/month for unlimited links. Currently, no export functionality.


Why would I want to save a click? The project is very cool and I will use it.


Oof, any time I see next/prisma I already know that my tiny VPS will likely choke building this... So yeah, self-hostable, but not for everyone.

Got burned with this by cal.com self-hosted version: https://blog.vasi.li/cal-com-is-making-me-lose-faith-in-the-...


Here's a (my own) lightweight alternative, built using django & no javascript: https://gitlab.com/sodimel/share-links

It allows you to store links (title & language of the page, a pdf of the page, assign tags, to include them in collections), it has a very simple (moderated) comment system, set status of the link (online: direct link, offline: replace link by a webarchive one) a lightweight ui (remember: no js), multi-accounts (permissions), translations, some rudimentary stats and some other things (access a random page!).

See my own instance for an example with thousands of links: https://links.l3m.in/


Same. I don't think I need the collaboration aspect of this app, so I will keep being a happy user of linkding, see: https://news.ycombinator.com/item?id=21872488


Actually Linkwarden was tested on machine with only 2GB of memory and it ran pretty smoothly.


Heh. Not a fan of js apps (npm or not), but your article was enjoyable to read.


Thank you.


Build it on your own computer, rsync the result to your vps


I'm making a similar thing with SvelteKit and Kysely so we'll see how that turns out.


I have saved this to my bookmark manager! :-)


This looks slick. Because archive.org is getting a little problematic by not allowing more sites to be archived, decentralized archiving is becoming more important. I have been using archive box on my server. It does not have the collaboration features, but that is what my fediverse instances and other collaboration tools provide.


> Because archive.org is getting a little problematic by not allowing more sites to be archived

I haven't heard anything about this, could you elaborate or link to some article?


I am sorry that details escape my memory at this point, but I have seen a couple instances recently where journalists tried to archive news stories and were served a response that someone has barred articles from this site from being archived. There is also no guarantee that something once archived there will not be removed when they are put under pressure or terms of service change.


So at one point the answer was robots.txt and now it's not: https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea... - that information appears to be current - email info@archive.org and request removal is the process, which some "reputation management" firms talk about. Weirdly I can't find much info.

Furthermore, I don't think archive.org tries to hide/obfuscate their user agent so it's relatively easy to block them - I know that it's possible to manually upload stuff to archive.org, and there are other sources (partnerships with Cloudflare and Brave, at a minimum) but that's not as easy as the Wayback Machine.


I’ve heard no such claim. Interested in an offices statement as well


Does this have the capability of setting an option to periodically check the page for updates and save a revision?

My ideal bookmark/page archiver would have this workflow:

1) Find a page I like or find valuable for whatever reason, so I click on a browser addon button.

2) A little dialog would then show up from the button, allowing me to set the following

2a) Add tags, as well as offer suggested tags I could add or remove.

2b) Set an optional update frequency, preferably with an option that would slowly reduce the frequency of checking for changes, first if no changes are found, and eventually as an absolute regardless of changes.

2c) Set specific technical page save settings

3) Once done, I click a “save” button in the dialog, and the page would be saved at a single html file, like the browser addon “SingleFile”, (which has some adjustable default settings previously mentioned). This allows saving pages with very simple javascript/dynamic functionality instead of essentially an static image. It also inlines some media: see https://addons.mozilla.org/en-US/firefox/addon/single-file/. That said, perhaps a WARC file may be better when it comes to handling things like compression, multiple revisions, indexing, and possibly following links to download and store linked media.

4) Then it would automatically open the saved page in the browser, so I could have a quick look can make sure it’s not broken for some reason

5) Finally it would then occasionally check for updates, saving a revision. On future visists to the page, the addon would have a little badge to let me know the page has already been saved and is being watched.

It kinda sounds like I want a browser integrated front end with sane and intuitive settings for HTTrack. As and example, let’s say I find a post on hackernews full of insightful comments about something and want to save it. The post might be new, so comments are going to continue to be added (or possibly removed, though this is more of a reddit problem) after I’ve saved the link and page. It’d also be nice to automatically grab the linked webpage for the context. Something that makes this easy would be great.

It might also be nice to be able to select comments (select elements like ublock does?) for highlighting.


Other things that come to mind as complications.

Saving the page as presented in my current browser session can be vastly different vs a non-logged in guest with no changes from browser addons.

Many websites require browser addons to be tolerable. Reddit likes to hide the end of comment chains to artificially inflate their fucking click metrics, and addons are required to load those comments inline. Saving pages with ublock enabled is also a must. I think selenium can do this: https://stackoverflow.com/questions/52153398/how-can-i-add-a...

So being able to use a login token or auto login with an would be useful. It’s probably best to create a special archive only user for each website. Otherwise it’d be a nightmare trying to remove the elements such as username, favorites, subscribed, etc and make sure the redactions aren’t broken by a future site design update.


I suggest trying out HamsterBase (HamsterBase is not open source).

1. It supports direct binding with SingleFile, enabling one-click web page saving. Because it saves in the browser, all other plugins will take effect.

2. It provides an open-source plugin https://github.com/hamsterbase/hamsterbase-highlighter, allowing you to annotate directly in the browser, and it automatically saves a snapshot of the web page when you annotate. When you visit the page again, it automatically displays the previous snapshots.

3. All data is stored on your local device, with both a docker version and a desktop version available. Different versions support P2P synchronization.

4. Provide full-text search function, which can search all the articles on the webpage.


When will docker version arrive?


Precisely when it means to.


This is great! I've always wanted something like this, and even tried launching my own app for this. My app failed to gain traction but happy to see the same idea succeed.


Just a bit of advice: You wrote a sentence about what the service does, and a large paragraph on what it was built on. When you're pitching your service, tell people what's different about your service, why it's better, why they'll want to use it, etc.

I understand that HN tends to be more technical, but the technical details can be a single link. Right now, all I know about your project is that it's a bookmark manager and S3 is better for storing files than the filesystem.

Good luck!


I threw together a quick docker-compose.yml to try this out. For anyone else that wants to spin this up and give it a try locally: https://gist.github.com/joekrill/cc503e21e14f95fefa91acc5f86...

(Though I'm seeing crashes on my Mac M1, which I think is being caused by Prisma)


Looks really clean.

A few questions:

- It’s not clear if this saves highlight in Ng and annotations (notes about the highlights). More than saving a bookmark we think about a sentence that can be searchable.

- Is there any plan to save the entire webpage as text (to maintain the annotations in it) in addition to pdf and screenshot?

One product I am overly dependant on is Diigo - I would love a replacement even if it was self hosted.


Saving webpages as text was actually something we wanted to do before launch but just went for the “MVP” for now.

So yeah we’re definitely bringing more archive formats.


Please consider integrating SingleFile support as well, perhaps at least in coordination with the planned browser addon. (ArchiveBox supports SingleFile on the server-side running under nodejs.)


Great to hear, will keep an eye out for it. I take the saved pages and pull them through my workflow via API.


Hey, this looks great!

In your readme, in the "A bit of history", it should be `has many fewer features`

On a more technical note, I wondered if you have any stories working with Prisma and Next? It works but every ORM has its pros and cons. My annecdote with the two is on a project recently, I had issues bundling the appropriate prisma packages during a Next standalone mode build.


Prisma is great and I definitely recommend it to anyone who’s either starting out or on a more advanced level.


In addition to PDF and PNG, died it store searchable text from the page.

If really live a way to do a text search of my bookmarks.


I wonder. Combining this with ocr in obsidian or something might get you there. It’s not a single tool solution though.


Cool project, quick design feedback, in 'Exploring the use cases' the left column is too narrow? https://ibb.co/f4Q5mnB


I've used Raindrop[1] for the last few years and it works well - cross device support, archived pages, and tags/folders.

Going to check out Linkwarden since I really like the idea of being able to self-host something similar since Raindrop could one day disappear (#googlereaderneverforget).

A feature Raindrop has is it can export bookmarks to a standard xml file, which I then have a script that automatically adds them to Archivebox[2] for a local copy and to add them to archive.org[3].

Does Linkwarden, have a feature to automatically submit a bookmark to archive.org along with the local copy? That would greatly reduce this setup and have it all in one tool.

1. https://raindrop.io/

2. https://archivebox.io/

3. https://ecliptik.com/bookmarking-with-raindrop/


How has your experience with archivebox after running it for a while? After trying to set it up multiple times I gave it another try a few days ago and it always feels like it's doing too much and is therefore very sluggish and buggy.

I was looking for alternatives but couldn't really find something great with a decent UI and full-text search.


It isn't horrible. I have it running in a docker-compose stack and after initial setup I haven't really thought about it other than checking the Raindrop script I have is still populating it.

I don't really use it interactively, it's more to have a "backup" of websites I find useful after finding some I used to reference for years disappeared and were never added to archive.org or occasionally sending the Readability/PDF versions to my Kindle.

I also setup YaCY[1] at one point with the idea of having my own local personal search engine for the archived sites, but I ended up never using it.

1. https://github.com/yacy


Similar story, getting ArchiveBox setup and running was a breeze, but everything after that was kinda rough. For one, ArchiveBox doesn't have a proper API, so I had to rig one up with Puppeteer. And then there's YaCY. On paper, it seemed like the dream tool for indexing and making a searchable bookmark collection. But in reality, it was a whole lot of work followed by a whole lot of disappointment.


I’ve tried Archivebox (using docker compose) several times, and every single time it just stops.

I import around 3k bookmarks, it starts archiving them. Immediately some archival methods fail (usually screenshot and pdf), and after archiving a few hundred bookmarks it never continues to archive the rest. I‘ve let it sit and do its thing for several days, it never manages to get through all of them (or even a sizable minority).

Different machines, different filesystems, different networks. No idea what‘s wrong.


I’m glad to read that as it confirms my experiences too. Seeing that it is also not that actively maintained I even started writing a similar thing myself as I really only need a small subset of the functionality.


Being able to bookmark a Link to archive.org was actually something we wanted to do earlier, but we had to do it a opt-in solution per each link since there might be a website that you don’t want to archive for the public and instead only keep it to yourself.

But note that it is on the roadmap (but not top priority).


Can it import a list of URLs and aut-tag them using some API or pre-trained ML? If yes, I bloody want it! No matter the price.


>Easily share curated collections with the public

Do you have a page that shows the most popular collections?


I am always curious why don't people use zotero directly for archiving everything


Is the browser extension coming for Safari as well? And what about an iOS App?


Where's the documentation? I get page not found eror


Are there full-text-search capabilities?


If by full-text-search, you mean the website contents, not really.

But if you mean, searching the link details, yep.


If you want full-text-search with archiving check out my project, DiskerNet. https://github.com/dosyago/DiskerNet --> also well done on LinkWarden! Looks like a great product! :)


Not to diminish the effort here, but I just want to point out (as someone who has tried lots of bookmark managers) that Floccus is everything I want from a bookmark manager (effortless sync across devices and just using the bookmark manager in your browser).

I am pointing this out, because I wish someone would have pointed it out to me.

https://j11g.com/2023/03/04/floccus-is-the-bookmark-manager-...


"collaborative" is the key feature that Floccus and all other "syncing" bookmark managers are missing.


Missing features: a good UI for managing and organising bookmarks, automatically archiving bookmarks in case they go offline


Another reason why Safari shouldn't be considered a user friendly browser.


Is this simply because the bookmark manager linked (Floccus) is not available for Safari?

Or better yet, can you elaborate on how any of the content up the chain from your comment that shows why Safari shouldn't be considered a user friendly browser?


People who consider Safari a friendly browser do not look before iCloud and do not look after iCloud. No matter how it behaves/performs. That’s how Apple ecosystem rolls. And the ones who do not consider it a user friendly browser do not use it.


I’m open to an answer to GP’s question but this wasn’t one


Which reason are you referring to?


The fact that you can't use this extension (amongst many others) with Safari - therefore Safari on MacOS and iOS/iPadOS cannot benefit from this type of sharing. Walled garden strikes again.

I'm not saying that Safari is a bad browser, but artificial limitations imposed by Apple on the browser and the OS is quite frustrating for me.


Safari supports the same extension standard as the other browsers, they even have a tool to convert extensions into the Safari format. All the bookmarks are also in an sqlite database which you can access, or export them as a file, this is not a case of a wallet garden.

I know because I did just that with my Firefox and Chrome extensions. The only thing that's keeping developers from doing that is that you have to pay the developer fee to publish the extension app, on top of the regular differences between the browsers that you have to take care of if you are building an extension.

https://developer.apple.com/documentation/safariservices/saf...


That's all great but neither this nor www.xbrowsersync.org supports Safari.

That's the reason I don't use Safari beyond random superficial browsing.

Here is what xBrowserSync has to say about it:

"Will xBrowserSync support Safari?

No and it is extremely unlikely this will ever happen due to Apple moving away from the WebExtensions API and forcing developers to purchase Apple hardware and pay $99/year to develop on their platform."


eeeeh.

Shiori looks like it'd work infinitely better compared to floccus. It has an extension, tags, and everything is stored in a central repository you can visit from web (or server itself) any time you want. It also archives your bookmarks. It has been working flawlessly for me for a couple of years now.

https://github.com/go-shiori


I've been self-hosting and using Shiori for couple years now. Originally it was my replacement for pocket. It’s the closest thing I found to Pocket. Not only for read-it-later, but also as my archiving tool. My only issue with Shiori is that I have not been able to find a working browser extension or get the official Shiori extensions to work. So adding URLs is a manual process. So I only use it archive dev and longform articles for reference later.


Really? The official extension works fine for me. What issue are you running into?


I couldn't get the extension working on any chromium based browser. (or chrome or edge either)

Using the chrome extension - I get an error "json: cannot unmarshal bool into Go struct field .remember of type int" when attempting to login to my local Shiori server.

The extension - https://chrome.google.com/webstore/detail/shiori-create-book...

Looking at the reviews on that extension,that seems to be the common complaint.


That looks like a web app, not an actual browser bookmark sync.

Apples and oranges.

There is a value in using native browser bookmarks and syncing them cross browsers/OS's.


>There is a value in using native browser bookmarks and syncing them cross browsers/OS's.

Don't most browsers do this automatically WITHOUT a third party app? Firefox and Chrome both sync bookmarks across devices. What is the usecase for a third party bookmark syncer in that case?

Shiori acts both as an archiver as well as bookmark saver. My bookmarks are ...cluttered otherwise. I have a OneTab page with over 37000 'tabs' saved.


They don't natively sync cross-browsers i.e. chrome/brave/firefox.

Safari would be a welcome addition but alas.


Genuine question, not trying to bash the project -- the link here seems to really stress that floccus is just for syncing, but can't you just use Firefox Sync for that?

I already have the ability to send my tabs across devices or sync bookmarks, it's built right into Firefox. The UI could be better, but it doesn't look like Floccus changes the browser UI, which is my primary complaint with Firefox bookmarks.

I'm not sure what I'm missing.


Thank you! This is exactly what I needed, and what I've been looking for for years! Open source, lightweight, and stable.


xBrowserSync is another one.

You can use https://github.com/ishani/xSyn for self-hosting.


I ended up just creating a page in Notion and imported a CSV file.


Thank you, seems like what I wanted.


doesn't look like i can use my own server with floccus


If you have a WebDAV server, you can use that.

I use several browser profiles (stuff like social, entertainment, dev), and now I can put the usual sites I visit with each of those in the top level of the their bookmark bar directly, but also have a single folder for the ones I want to share between all of them, yay! I am very happy right now. Thanks GP.


I like to save the best / most interesting links I come across as I browse the web. It can come in handy to pull up a blog post I read a while ago or remember some new sass product or developer tool I wanted to check out. I'm using https://raindrop.io now which works great for this.

When I looked into it I was surprised that browsers don't have this kind of bookmark management built-in. I'd be very happy with two small additions to browsers: (1) display by / sort by date added and (2) a small separate freeform text box for notes (so I can describe why I saved the link).

(Optionally it could be nice if browsers adopted some standard sync mechanism for bookmarks, maybe based on WebDAV like the Floccus extension).

Then again, these dedicated external bookmark managers do have nice features like tags, search, and offline downloads or page screenshots. Those are all great!

Linkwarden looks like a nice product. Looks like it would tick all the boxes for my use-case and the design is pleasant. I like that it's open source and has a fair price for the hosted offering. Maybe I'll give it a try!


Any relation to Bitwarden, or just a happenstance similarity in names?


It is a Linkedin for Bitwarden.


No please no. We have one LinkedIn, and that's enough pain to humanity.


No we’re not related to Bitwarden, we both just have a nice name and are opensource :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: