Show HN: JAQT – JavaScript Queries and Transformations

[Poefke]

https://github.com/muze-nl/jaqt

Hi all,

I've made a javascript library to simplify searching/sorting/filtering in arrays of objects. Its inspired by both GraphQL and SQL, but implemented using javascript Proxies. Instead of creating a new language, its all just javascript.

I've made it as part of an experimental database, which uses javascript as the query engine. The normal javascript map/reduce/sort functions are quite difficult to master for junior developers. JAQT is easier to explain, and can still be used in combination with any existing array functions.

Please let me know what you think of the API and its ease of use!

[jansommer]

Wouldn't the query for your example just be

data.filter(d => d.friends.includes("John")).map(d => ({name: d.name+" "+d.lastName}))

Maybe I'm missing the bigger picture, but that doesn't seem so bad

[allannienhuis → jansommer]

one of the cases something like this is useful for is when you have user configured saved 'filters' - you can simply pass the stored object into the where clause. I agree if you're doing hard-coded queries on arrays of objects, this may not add a lot. That said, a consistent syntax for doing searches and mapping is useful - projects like lodash have that as part of their api too.

[throwitaway1123 → allannienhuis]

This is actually a pretty decent use case for something like this. You could technically convert a user provided filter function to a string to serialize it, then recreate it using the Function() constructor, but that requires adding 'unsafe-eval' to your content security policy [1][2].

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

[allannienhuis → throwitaway1123]

and it requires users to define filter functions, which isn't happening with any users I deal with :)

[meiraleal → jansommer]

I liked JAQT API but it is not really good to create a layer on top of already simple functional constructors.

[austin-cheney → meiraleal]

Why is that not good? JavaScript is full of layers of unnecessary abstraction often for completely superficial reasons and sometimes even irrationally applied at great cost in defiance of evidence and simplicity. So, what makes this specific example less good than all the other countless examples of abstractions most JavaScript developers would happily die to defend?

[meiraleal → austin-cheney]

because in the past it didn't have those functional constructors so things like underscore or lodash would be needed but adding a layer as you mentioned. It is not good, for me, to add a dependency for this again. But in the end I have in my utils file a few functions that basically do what JAQT does and I might copy a few more.

[ath92 → austin-cheney]

jQuery provided a DOM manipulation API that was the same in all browsers when there were lots of differences between browser implementations. While you could do everything using standard web APIs, that would be incredibly error prone and tedious at a time when `querySelector` was not a thing yet.

Lodash does way more than some basic functional wrappers. I've seen too many buggy re-implementations of debounce, throttle and groupBy at this point, it's not even funny.

[austin-cheney → ath92]

Life without jQuery was never as hard as you claim. jQuery broke in IE9 more than it helped, but it was also a cult and many developers were entirely unemployable without it.

[eyelidlessness → austin-cheney]

This is such a weirdly contrarian take. You’re basically saying that everything in context is terrible, and somehow at the same time, on the same basis, that it’s wrong to critique anything because everything is terrible.

That’s certainly a logically consistent position, but it isn’t one that allows much room for anything to improve… or even become a productive discussion.

[austin-cheney → eyelidlessness]

No, I am saying don't complain about something only because its an abstraction when likely the person making that comment cannot program without vanity abstractions. Clearly, this is about bias and preference. At the very least people could be consistent in their reasoning.

[eyelidlessness → austin-cheney]

The critique was that the existing abstractions address the same set of problems as do the newly introduced abstractions. You’ve said that the existing abstractions are bad, as an invalidation of that critique.

It is perfectly reasonable to ask whether something for convenience adds value to something also for convenience.

Nearly everything in any high level language isn’t strictly necessary to achieve general purpose computation. That doesn’t invalidate any distinction between all higher level concepts! They specifically add expressiveness, so it’s eminently meaningful—for people who find expressiveness valuable—to discuss and distinguish between like solutions in those terms.

[austin-cheney → eyelidlessness]

I never said any of this is bad. I said it is completely unnecessary. Those are different. I also said the blind reliance on that unnecessary stuff is foolish. Like you said it is all about convenience, which is exceedingly shallow and faulty.

> They specifically add expressiveness

Not really. Expressiveness is the ability to achieve the same end state in different ways. How do you then determine that? You have to count the different ways a given thing is achieved. Under the abstraction if its doing the same thing in the same way as something else then its not really different, and thus not expressive. That is why these large SPA frameworks have wildly different super large APIs, but their output is strikingly similar, which isn't expressive at all. Really, with these tools none of that matters, because its really just about the perceptions of convenience.

[doctorpangloss → austin-cheney]

You’re right but arguing with the “add another dependency” people is a waste of time. This just isn’t really a software engineering community.

[meiraleal → austin-cheney]

This is a discussion forum, you know? Different people, different opinions. You can go ahead and use JAQT which I found fine but I prefer to avoid adding external dependencies. I hope it doesn't hurt your feelings too much but the good thing is that you can continuing using JS the way you like.

[Poefke → jansommer]

Author here, if you are comfortable with filter/map/reduce, than JAQT doesn't provide much benefits, I agree. My target audience is developers who aren't that used to that. I've had much more success with junior developers using JAQT, then trying to get them to use map/reduce.

[drawkbox]

I like that it mimics SQL queries/filtering which is straightforward.

Typically I use something like JSONPath [1] (basically XPath for JSON) [2] or jq for this but having more options with other syntax and style is helpful.

For streamed JSON like NDJSON [3] there are some nice filtering options [4]

[1] https://en.wikipedia.org/wiki/JSONPath

[2] https://datatracker.ietf.org/doc/html/rfc9535

[3] https://github.com/ndjson/ndjson-spec

[4] https://github.com/mbostock/ndjson-cli?tab=readme-ov-file#fi...

[ssahoo]

Looks like .net Linq. Many js projects use lodash and variants, how do you compare that?

[Poefke → ssahoo]

LINQ is much more sophisticated, translating as much of the query as possible to the target (server) language, like SQL. JAQT is not doing that. lodash, underscore, and others, had a great impact on javascript by introducing lots of functional paradigms. Most of that is now part of javascript itself. JAQT's main purpose is to allow you to filter arrays of objects and create custom objects, just like GraphQL. And be a bit easier to learn than Array.map/reduce/filter.

[csbbbb]

With JSON, sometimes you just need a fluent, permissive way to query objects that you don't already know a type structure for. This seems useful! I love JSONata for exploring json data, but it's more useful for static analysis than programming.

[gryzzly]

can a query like that be observable? so updating the array/object would cause the results to update?

[af3d → gryzzly]

AFAICT this library generates "flat" JSON objects. Wouldn't be too hard to implement it such that references to Javascript objects are instead returned, where some sort of callback could be invoked whenever the actual object is modified. I do rather like this "query engine" approach, too. It's a much cleaner interface IMO and doesn't seem to incur too much overhead either....

[throwitaway1123 → gryzzly]

Dexie supports observable queries, but it operates on IndexedDB rather than plain JS objects: https://dexie.org/docs/liveQuery()#vanilla-js

[jansommer → gryzzly]

I don't know if RxJS has run out of fashion, but that would give you observable js using map/filter/reduce etc.

[kayo_20211030]

Thanks. Looks really useful.

[iddan]

This is everything i wanted

[fbn79]

For the same purpouse I ofter use this https://github.com/calmm-js/partial.lenses

[e1g]

There is another excellent write-up exploring the theory / use-cases for such libraries https://www.inkandswitch.com/cambria/ (and a corresponding library at https://github.com/inkandswitch/cambria-project)

[porker]

In a similar vein is https://pbeshai.github.io/tidy/ which I've used for 3+ years. It's a really nice lightweight transformer.

I've also used https://github.com/uwdata/arquero once (better performance for large datasets).

[pdyc → porker]

any idea what does tidy uses internally? or is it iterating over entire array for every operation?

[ivanjermakov → FractalHQ]

This is an amazing use-case for TypeScript's conditional types: https://www.typescriptlang.org/docs/handbook/2/conditional-t...