The volatility calculation looks like it should be doable in q/k, I'm not sure about the more complicated stuff but at the end of the day it's a general purpose language too so anything is possible. KDB being columnar means thinking in terms of rows can often be slower. Sounds like you have a keyed table? If the KDB guys you have aren't that good/helpful you could check out some other forums. Could be useful for the future to be able to circumvent issues you have with the kdb devs.
Doesn't work on empty arrays, whereas the K does and gives the reasonable result of 0 (for total water) and -inf for maximum height. Maybe it doesn't matter, but sensible defaults are a good thing!
Edit: didn't look at the error closely enough to see that the only thing that breaks is the maximum height. That's not that bad, but still not ideal imo.
The result may be reasonable, but this is just a fluke.
Numpy complains that: "ValueError: zero-size array to reduction operation maximum which has no identity".
This error makes sense. The only meaningful result for the maximum of an empty array would be minus infinity. It should not be zero. If it's zero, it means the programming language makes the silent assumption that the elements are non-negative.
I very much prefer to have the code throw back a meaningful error at me than a reasonably looking result, and then fail silently in a different situation.
Would you agree with the statement, "the maximum of an array should be contained in that array?" That seems like a reasonable constraint that most languages uphold.
No. I think the maximum of (x:xs) should be equal to max x (maximum xs). (Using haskell notation). Logically this implies maximum [] should be -infinity if it is defined. On finite lists, maximum is also the same as the supremum, and on infinite sets the supremum doesn't necessarily have to be contained in the set. So there's at least some precedent for not having the maximum in the list.
Going by your definition, max (x:xs) = max x (max xs) = max x (max (head xs) (tail xs)) = ... ad infinitum. The basic problem with that definition is that it only really defines max for non-empty lists, otherwise the pattern-matching against (x:xs) will throw an exception.
In mathematics maximum and minimum of a set are elements of said set. So the empty set has neither a maximum nor a minimum.
There is another pair of terms that obeys the rules you want max and min to obey: supremum and infimum: <https://en.wikipedia.org/wiki/Infimum_and_supremum>. And indeed supremum of the empty set is -inf. And infimum of the empty set is +inf. That’s because supremum and infimum of a set don’t need to belong to said set. They only need to belong to the set of bounds (upper for supremum; lower for infimum) of said set.
That definition strikes me as intellectually satisfying but less useful for engineering software systems around (for the same reasons as the other commenter). But maybe it's more appropriate in the domains K is used for, I wouldn't know.
If you want a serious answer, as a k programmer ||\|x (which I assume your example is based on) is instantly readable as producing a bitmask of ones up to the last. ie 0 1 0 0 1 0 -> 1 1 1 1 1 0. Or a max scan from the right on integer arrays. I don't know how to even describe that in English succinctly, let alone most mainstream programming languages. (Haskell - reverse.scanl1(||).reverse - ok, not too bad)
You've sort of answered your own question. None of those three would obviously mean the same thing to me. And that's just one possible combination. If you consider all possible combinations of 4 symbols in K, most of them will have a distinct but obvious (and useful) meaning that is extremely hard to summarise in a single word.
But to me as a relatively experienced APLer, it reads very clearly as "return if there are no non-ASCII chars in `w`" (and if there are, the indices of those chars will be stored in `i` and the code point of every char will be stored in `c`).
In case anyone is curious about if the system is still used 12 years later, the latest thing I could find was from a job advert https://app.verama.com/job-requests/40623
"This application is built in using Dialog[sic] APL which is a niche language and has its own challenge and that’s where second application Apollo comes into the picture which is created by another team Atlas aiming to create the Hercules using updated tech stack together with new functionalities."
I wonder how successful the replacement will be... inb4 end users will complain forever about shitty UI because they replaced fast 3270 with hunt&click gui[1] or slow website
[1] Long ago, trying to deal with an issue related to not being able to make SWIFT transfers from my then-current bank, I ended up going to branch myself, and while trying to fix it, struck a conversation with the elder teller lady. She had words regarding "ergonomics" of the win32 UI that replaced her block terminal.
Pretty sure it's impossible to calculate unique/group in O(n), unless either the data is pre-sorted, or your range is extremely limited (and problem says nothing about this), so it has to be at least O(nlog(n))
For K, I think the solution is to find unique first (nlog(n)), the for each unique value go over the list and find how many input values match (multiply by n). That's O(n^2 * log(n)), even worse than I wrote initially.
No, the k solution computes the unique elements and group independently. #:'=v[;1] uses group for the first column, and ?v[;1] uses unique for the second. (;) is list notation.
Your analysis of the algorithm you've described is also wrong: it would be O(n*log(n) + n^2), or O(n^2), because the second part loops over unique elements and not operations performed by unique.
In the random-access model, hash table construction/lookup is O(n) average case for n elements, if the elements have bounded size. More generally, it's linear in the total size of the input. If the strings are very long it can take a while to hash and compare them, but there's never a log(n) factor. Maybe you're confusing this with results from comparison sorting?
Both solutions are O(n) with the assumptions of random-access memory and bounded string size (or hash/comparison time), which I consider to be pretty standard.
If numpy/matlab/octave/... was 'all the same stuff' with none of the downsides and all of the benefits, why would k or array languages in general have such a passionate community? Either every arraylang-er is masochistic (only slightly true) or there's more to it than you think.
That's a weird argument... passion is not always connected to efficiency/practicality/usefulness, in fact I feel that in a lot of cases it's the opposite.
As a great example, look at old computers: they are pretty popular (there is one link on HN front page right now about N64!) and people are super passionate about it, and yet you would not recommend "everyone to play at least one Nintendo 64 game".
For non-computer examples, somehow a number of my friends are extremely passionate about super hot (as in spicy) sauces. They bring a new bottle for every meal, and they can talk about the kinds of peppers for hours. It's interesting to hear, but I would not want to try them myself, nor would I recommend this to everyone.
I'm sure people who are into retrocomputing or spicy food or any other topic could tell you why they prefer that to something more mainstream. Whether or not you would agree, you could recognise that they had different opinions. The issue I had with your comment was writing off k as simply an inferior version of numpy etc rather than trying to consider why people like it, and what it may even do better than the mainstream alternatives.
> Learning array languages makes you a better programmer in any other language. The way you think changes. We are taught to approach a problem procedurally. Actually the for loop is your enemy.
I happen to strongly agree with most of this: learning array-based "operations", writing the code where "the for loop is your enemy" is a very valuable experience, and if will make you a better programmer in any language. Everyone should try writing some code like this.
The problem is K is a mix of three things: array-based operations, point-free style / tacit programming, and extreme brevity via punctuation abuse and lack of types. And not all of them are equally useful.
It's like trying to introduce someone to Chinese cuisine and giving them the spiciest Hunan dish there is. Unless the person already happened to love spicy food, the chances they will just write-off the entire cuisine.
I bet there people today who associate "array programming" with obscure character soup of APL/J/K and ignore it because of that. Who knows, maybe if they knew that numpy/octave offers the same thing in more palatable package, there would be more arrays and less "for" loops in the world.
The original comment said learning "array languages". There's some debate about what that means, but generally it means APL/J/K/etc not numpy/matlab/etc.
As far as I can tell (correct me if I'm wrong) you haven't learned an array language, yet you're telling someone who has (and anyone else who reads your message) not to bother.
To use your food analogy, this is like saying "Don't try real Chinese food in a restaurant, just cook some rice at home. Same thing."
It's not the same thing. The flavour is completely different.
What would you see as the big benefit of array languages over simply array-based programming in other languages? Or in other words, what sorts of things will I learn from trying out J or APL, that I wouldn't learn by playing around with Numpy or Pandas?
In addition to what rak1507 said, knowing what parts you’re composing gives you a more intimate feel for how the pieces fit together. Knowing that you’re building on top of a handful of functions you probably already have some intuition for is quite different from using a library function and simply trusting that the authors have done their due diligence.
It's hard to think of concrete benefits on the spot when the biggest benefit is the somewhat nebulous "feel" of the language. But generally: array languages typically make use of composition of primitives rather than larger, more complicated built in functions. For example in k, there is no 'sort' primitive. Instead of sort, you have grade (<). In APL this is ⍋. In numpy this is np.argsort
At first this is a bit counterintuitive. But then you realise, x@<x is sort. y@<x is sort by. <<x is 'rank'. There are all these 'recipes' that are based on <, and some of them you may not discover using numpy, as there would be another function that did it for you. You don't typically solve things by composing simple elements, as that would be far too verbose, instead using complicated built in functions, often with many parameters to describe the desired behaviour.
I'm sure other people will have other opinions on what the benefits are, but that was just one example off the top of my head.
I've read books on APL, J, and K and have played around a bit with all of them for some toy problems. I've also used Python's Numpy & Pandas at work.
The APL/J/K and Python + Library philosophies for programming are super different and if this doesn't make sense to you, I'd recommend downloading Dyalog (it's free to play around with) and go through some tutorials and see if you still feel the same way.
Yes it's all programming like how rice and french fries are both food. They're both awesome too. With Python you have a general purpose language that lets you build up a matrix in a manner you're used to. You've then got an object you call methods on to delete columns or update an entry or whatever. With APL, you've got what is like a hyper optimized DSL for doing that. I'm a lot more proficient with the Python option and it's both popular and free, so it's probably the only option I'll ever have to use at work. Using the APL/J/K options are a ton of fun and I think I could get hyper efficient with them as well given the time. Using the APL is so alien compared to what I'm used to though. The idioms (e.g. ohh you want to do that...just use an outer product which is these two characters combined) are just so different than how I'd typically think about solving the problem (e.g. I'd use a nested couple of for loops or whatever). It really does change the way you think.
Off topic: I work with a lot of extremely large sparse matrices though (like 40,000 x 40,000 with most being 0s), so I'm not sure if APL is as optimized here and as a result it's also easier to stick to Python.
You can do 'regular' stuff in array languages too. And distributed searching/sorting sounds like something array languages would be good at anyway. kdb being the obvious example. The IPC in q for example is very easy to use, and seems better than most languages that aren't Erlang.
Well, I agree in the absolute sense. Not in the practical sense. In fact there is a ranking because on natural affinity and baseline-existence-of-what's-already in the domain. That's is what it is ...
FWIW I think array language communities are generally very friendly and helpful too. Not many programming languages/companies employ people to help people in chatrooms.
