On: ArchiveBox is evolving: the future of self-hosted internet archives

[nfriedly]

I haven't looked at the on-disk format, I just use the browser interface. (It's fairly common for me to save something from my phone that I'll want to review on a computer later.)

Here's an example of an Amazon "review" I recently archived that has instructions for using a USB tester I have: https://readeck.home.nfriedly.com/@b/tCngVjkSFOrCbwb9DnY2yw

And, for comparison, here's the original: https://www.amazon.com/gp/customer-reviews/R3EF0QW6MAJ0VP

It'd be nice if I could edit out the extra junk near the top, but the important bits are all there.

[ashildr → nfriedly]

I was about to post a link to the same URL but archived using singleFile, which looks like the original at amazon. I didn‘t because I realized that I have absolutely no idea what additional information would be hidden in the file. In the worst case any component sent by Amazon and archived into the file may contain PII, even if I am “logged out“.

I‘m not saying that singleFile is bad in any way, I‘m using it a lot on multiple devices, but I‘m not sure whether sharing archives is a good idea™.

[nikisweeting → ashildr]

100%, this is the challenge of archiving logged in content.

It becomes un-shareable unless we use fake burner accounts for capture, or have really good sanitizing methods.

[ashildr → nikisweeting]

Even when I‘m logged out I expect at least information on my geographical location to seep into the archive via URLs addressing specific CDN endpoints or similar mechanisms.

[nikisweeting → ashildr]

Yup, this is why the ArchiveBox browser extension sends URLs to a separate server for archiving with an isolated burner profile.

I should write a full article on the security implications at some point, there aren't many good top-down explanations of why this is a hard problem.

[ashildr → nikisweeting]

I know it’s a lot of work but this would be great and it may give readers a deeper understanding into security in general.

[ninalanyon → nikisweeting]

How does it save pages that are only available when you are logged in such as social networking pages?

[nikisweeting → ninalanyon]

You set up a chrome profile for archiving that's logged into all the sites you want to save. I recommend using burner accounts dedicated to archiving, so you'd have to add them to any private pages/groups you want to archive.

It is possible to use your main account for archiving but there are security risks (you cant share the snapshots without leaking session headers).

[pbronez → nikisweeting]

That’s a very cool solution- gives the user explicit control