Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Store.js v2.0 – Cross-browser storage for all use cases (github.com/marcuswestin)
162 points by marcuswestin on Feb 28, 2017 | hide | past | favorite | 54 comments



Store.js was originally released on HN in 2010 :) https://news.ycombinator.com/item?id=1468802

It's live on tens of thousands of websites (like cnn.com!) and has seen lots of improvements over the years.

Store.js version 2 is a full revamp with pluggable storage (it will automatically fall back to one that works in every scenario by default), pluggable extra functionality (like expiration, default values, common array/object operations, etc), and fully cross-browser automatic testing using saucelabs.com.

Feel free to ask any questions! I'm going to sleep now, but I'll make sure to answer in the AM.

Cheers!


Hey Marcus, I'm a user! Thanks for doing the heavy IE lifting in store.js.

Anyhow, I had a requirement to store raw strings to the local storage or userdata rather than go through your calls to JSON.parse/stringify. I forked and am using my fork rather than your maintained version, which is sad. Since you've got the hood popped for maintenance, would you mind adding raw string support to store.js in version 2? Thanks in advance. Here's my fork repo for reference, please see the getRawString and setRawString API's I added:

https://github.com/woldie/store.js


Genuinely curious but why is this important? Assuming the current API only takes an object can't you just provide 2 functions

    function putString(id, s) {
       storePutAPI(id, {s:s});
    }
    function getString(id) {
       var o = storeGetAPI(id);
       return o ? o.s : undefined;
     }
Note I have never used the API nor looked at the docs but assuming you can store objects why the need for special functions?

On top of which AFAIK JSON.stringify and JSON.parse work with plain strings so I'd be curious what prevents using plain strings in store.js without any changes

Sorry this isn't a put down , just trying to understand


Not at all at all, I understand your confusion. Mine is definitely a niche case, perhaps straying a bit into the eclectic.

I have put together a virtual file system for a programming environment for kids that is built atop the local storage/userdata API's.

I support legacy IE back to IE6 as well as modern browsers, and I wanted to maximize my very limited allowed storage space on legacy IE.

Also, I needed to have some certainty around how much free space is remaining in the file system. My design was to organize the underlying storage into 4K blocks that were pre-allocated when the system first starts; whatever the host system could give me in terms of space I would take.

JSON is nice for the general case, but the way it escapes some characters makes computing the free space I have remaining unpredictable as new files get written.

Of course, I can do my own encoding of bytes into unicode char codes and write those as raw strings using store.js. JavaScript strings are saved as 16-bit unsigned arrays if you unfocus your eyes and stare at the screen for awhile, and I use as many bits as I am able in my scheme.

So, I rigged up a byte-to-string encoder/decoder that was universally accepted on IE and modern browsers, and calling through to my getRawString/setRawString API, I just about doubled my storage over the default JSON.stringify/parse!

Suprisingly, performance isn't bad with my encoded strings either. That said, my compy has a retro flair, and I don't think my users will mind much that they have to wait a second or two to load their games from disk. Perhaps even adds a bit of authenticity. :)


Bonus points for best use case on the page :)))

Eclectic as it is, I think we should totally consider making it a plugin/storage! If you open an issue with links to the relevant material I'd really appreciate that. Thanks!


Hi! Allowing users to add the plugin behavior they want is the whole idea, so let's do it :)

Could you please open an issue? If you include a short snippet with the rationale for what and why you need it that would help guide the discussion.

Thanks!


Will do soon. Thank you Marcus!


Had somehow missed that this was a thing, it looks amazing.

Also kudos on https://github.com/marcuswestin/store.js#version-20 that is how to to do a readme page.


That's really nice to read :) thanks!


Will it support WebSQL in future too?


I don't know when WebSQL would be better than other storage alternatives, but yes, you could easily add WebSQL storage :)

The rationale behind the new architecture is to make it trivial to write new backing storages - see https://github.com/marcuswestin/store.js#write-your-own-stor... for a quick explanation of how you'd do that.

This is also true for adding new functionality, which sits "on top of" the storage (and is agnostic to whatever particular storage is being used). Here's a list of plugins that come out of the box: https://github.com/marcuswestin/store.js#list-of-all-plugins, and here is a quick explanation for how you'd write your own plugin: https://github.com/marcuswestin/store.js#write-your-own-plug...


If I'm not mistaken, your API is designed to be used synchronously. Not sure how WebSQL or IndexedDB could work towards a synchronous API.


That's also the way I read it, and indeed both WebSQL and IndexedDB only have async interfaces. IndexedDB used to have a sync interface available inside workers, but that was removed from the spec as there weren't any supporting use cases.

Overall localForage seems like a better cross-browser storage system, it also has a pluggable backend system (drivers) but defines an async interface.


Duh, of course! Sorry, 3am and not too sharp in the nut at the time :)

I'm genuinely interested in supporting every needed use case - would you be up for creating an issue? Either way I'll have it in the back of my head and see what can be done.

Cheers, Marcus


Does it support indexdb, or is it not designed for larger storage needs?


You can back it with any storage you want :) See https://github.com/marcuswestin/store.js#write-your-own-stor... for a quick explanation of how you'd do that.


I don't think so. Your API seems to be synchronous, where all modern IndexedDB/WebSQL/etc APIs use promises. The reason they do this is that in modern browsers storage often runs on a separate thread and/or use sqlite and you don't really want to stop-your-js-world when the rendez-vouz and sqlite processing can take 100s of milliseconds on mobile devices.

This really is one (of many!) of the ugly parts of modern JS development. Newly designed APIs will probably gravitate towards using Promises (because these also support sync backends - the other way around: not so much).

But, hell, it feels stupid having to allocate a closure to memory (and decorate your code with indentation and an additional function if you don't transpile using await) every time you just want to get your session token from storage.


Duh, of course! Sorry, 3am and not too sharp in the nut at the time :)

I'm genuinely interested in supporting every needed use case. Off the top of my head, I could see e.g get returning a promise in some cases, or passing an optional callback to get, or having a get.async... Either way - I'll have it in the back of my head and see what can be done.

If you open an issue on GitHub we can continue there. Cheers!


Marcus, when you say "Cross-browser storage for all use cases", I want to store media files for offline playback. Can I use Store.js for that? (TIA!)


I'm genuinely interested in supporting all use cases!

If you create an issue at https://github.com/marcuswestin/store.js and describe your use case in more detail we'll figure out what the right answer is.

Thanks!


Does store.js support saving files locally too? For example a PDF or similar?


I created an issue to investigate viability and usefulness :) https://github.com/marcuswestin/store.js/issues/193


Since I assume both have their pros and cons, when would I want to use Store.js, and when would I want to use localForage[0]? I'm asking as someone who has offline storage planned for an app, but has yet to get around to it. I'd like to know which one is probably best for my needs :)

[0] https://github.com/localForage/localForage


Totally! I created an issue to pull this info together: https://github.com/marcuswestin/store.js/issues/189

Check back in coming days for more details.

Store.js will soon have pluggable support for localForage (either by implementing similar functionality in separate storages, or by simply adding a pluggable storage which uses localForage under the hood).

There are a few benefits to using store.js, such as full cross-browser support and useful plugins, but the biggest meaningful value-add over localForage is probably the availability of a synchronous API that allows you to read and write values without callbacks, promises, etc.

In my experience you only want to use asynchronous APIs when you have to, since they add a fair amount of complexity, are a common source of errors, and are harder to debug properly.

But yeah, check back later and you'll likely find that store.js supports the same storages as localForage, in addition to the its other functionality.


Thanks, I'll keep an eye on that issue. I still have other more pressing milestones to reach anyway ;)


Sounds like (based on other comments) you should really just be using localForage. Store.js only currently supports a sychronous interface, which will be incompatible with modern web APIs.


On the other hand, synchronous interfaces tend to be less error prone and simpler to work with (and more enjoyable...? :)

There is an active issue to add the async indexdb/websql storages to Store.js (https://github.com/marcuswestin/store.js/issues/181) so that they can be used when needed (e.g when you need to store many tends of MGs of data), but for 90% of your use cases the simplicity of the synchronous store.js API may likely save you a meaningful amount of code and callback/Promise/stack trace debuggery :)


If you're writing buggy async code I'd say the storage library is far from your biggest concern. I'm all about human factors and making devs lives easier and all that, but async is a core skill for a web dev.


Store.js is here to make life easier for all web devs, beginners and pros :) I've concluded that adding an async API in addition to the sync one for access to the more powerful storages makes total sense though! Off the top of my head I'm leaning towards new methods for the async versions, and then probably adding localForage as a new storage.

Thanks for taking the time to make the case!


Personally, I wouldn't use a library at all unless I had to support IE<8. Localstorage is supported enough across browsers:

http://caniuse.com/#feat=namevalue-storage


I think something like store.js ends up being genuinely useful for anyone building functionality that depends on localStorage. More often than not you find yourself either having to deal with issues that store.js solves for you (E.g not breaking your site in Safari private mode) or requiring extra functionality that already exists as a small store.js plugin. (E.g expiring keys, default values, etc). All of these are things you can of course do without a library, but at that point you're reinventing the wheel a bit and not benefitting from all the automated testing that ensures cross-browser correctness. With that said, I agree there are times when just a quick read/write directly to localStorage can be the right call, but by default I think it comes back to bite you one way or another more often than not :)


Well, I need more than 5 MB and would like to store other values than strings (mostly large TypedArrays). Something simplifying my life there would be appreciated.


JSON.stringify/JSON.parse before getting/setting from localstorage.

The other requirements localstorage alone can't address.


It's worth noting that vanderZwan was asking about localForage, the library by Mozilla, not localStorage, the browser feature.


I'm aware of that. I'm saying if that your use case is simply to write to localstorage then a wrapper library is unnecessary.


I just upgraded code in my app (https://commando.io) that stores some user preferences from $.cookie (jQuery Cookie) to Store.js. Worked flawlessly.

As a bonus of switching from cookies to local storage, now requests don't send along those values anymore to the server.


This looks really good. I've been using PouchDB for doing something similar, but if I didn't need the fancy sync'ing I'd be all over this.


I use LocalForage if I need simpler solutions than PouchDB.


I believe syncing could live in a plugin! Would you be up for creating an issue and quickly describing your requirements? I'm genuinely interested in supporting every real use case!


Thanks for the work you've put into Store.js.


Thank you :)


localStorage is pretty much only thing we need these days.


Almost! Except if you want to store objects/arrays/numbers (localStorage only supports strings), or want your site to work in Safari Private mode (where localStorage breaks), or if you have to support legacy environments with old browsers (like many government services), etc etc etc :)


Not that I disagree with your other arguments but this one sticks out:

> Except if you want to store objects/arrays/numbers (localStorage only supports strings)

My normal usage of localStorage usually looks like this:

    const getItem = (item) => { JSON.parse(window.localStorage.getItem(item)) }
    // and
    const setItem = (item, val) { window.localStorage.setItem(item, JSON.stringify(val)) }
And then you won't have issues with any type.


Except when a value has already been stored previously without JSON.stringify (in which case JSON.parse will throw), or you need a default value if unset (e.g `store.get('username', 'anonymous')`, or ... :) But I also see your point that sometimes the very quick and simple solution can be useful!


Not if you want to store more than few MB (5 or 10 max depending of browsers if I recall). Not if you want/need an async API. Not if you need to be able to search your storage.


Yup on all accounts, although: > Not if you want to store more than few MB (5 or 10 max depending of browsers if I recall)

There's a case for something like http://pieroxy.net/blog/pages/lz-string/index.html

which can save up a lot of space


Let's create a plugin that automatically does compression!


Created issue: https://github.com/marcuswestin/store.js/issues/183

Will tackle a bit down the road.


I've found great value in simpleStorage, for those who are looking for easier object storage in localStorage.


Yes, simpleStorage is good too!

I believe store.js addresses all the use cases of simpleStorage, and then some (E.g Safari private mode, etc).

If that's not the case I will make it work for your use case :)


The only problem I've noticed with simpleStorage is the Safari private mode thing, but I see that as a bug in Safari so I just show a message to users telling them why the site won't work properly for them.


Does anyone have an angular2 version of this? Could really use it on something I'm currently working on


Hey, I'm not sure what an angular2 version would look like or do. Could you please open an issue with a more detailed description of what you need? I'll help if I can!

Cheers




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: