The problem you'll run into for any application of quantum computing to large language models is that quantum computers just aren't very good at big data applications. There's two reasons for that:
- Current devices, as well as devices likely to be built in the near- to medium-term are quite limited in the number of qubits that they implement. The current record for the most fault-tolerant qubits in a single device is 1. That's a hell of a lot better than where the field was at a couple years ago, but it's far from the huge amount of data that needs to be processed for LLM training and evaluation.
- Even if you have enough qubits to store training data, looking them up on a quantum device is still challenging due to what's sometimes called the qRAM problem. It's not trivial to make a quantum oracle that returns the data stored at a given index, and it's still an area of ongoing research to figure out how to do that.
That's part of why you see quantum algorithms being developed less for big data tasks and more for big compute tasks like chemistry. There, the program might be very large, but size of the input that has to be stored within the quantum devices and the size of the output you measure back out are both quite small, even down to a single floating-point number in some cases.
(source: I've worked in quantum computing for about twenty years now.)
That's why I emphasized one _logical_ qubit. I'll definitely argue that fault tolerance is necessary to achieve useful results, as you say, but there is some argument in the research community on that. Even setting that discussion aside, there's absolutely no way to run something like LLM training dirctly on physical qubits (unless there was an improvement in error rates that's probably on the order of 10^15 to 10^18), even if you had both enough to do so and had a good qRAM implementation.
> In some of its applications, the original
> Zeng-Coecke algorithm relies on the existence of a quantum random access memory (QRAM) [22],
> which is not yet known to be efficiently implementable in the absence of fault tolerant scalable quantum
> computers [1, 7]. Here we take a different approach, using the classical ansatz parameters to encode the ¨
> distributional embedding and avoiding the need for QRAM entirely. The cost function for the parameter
> optimisation is informed by a corpus, already parsed and POS-tagged by classical means.
Following my intuition, i.e. as an outsider that has been watching the progress of quantum NLP since 2012, I see the current academic situation in quantum computing as in the process of merging two branches, one being the traditional quantum computing field with concerns stemming and application thought in mathematics, computing theory, physics(and upwards chemistry->biochemistry->biology), the other branch being a fork carried out by Coecke (quantum logic), Abramsky (computer science) and Sadrzadeh (epistemic logic) who saw in categorial formalisms of quantum logic a way to mix compositional (syntax, logical rules) and distributional (statistics, "bag-of-neighbor-words") representations of meaning. In this regard they bring new methods but also new applications of quantum computing, with a focus on NLP, as language given this "natural tensor structure [20, 35, 23] [...] can be considered quantum-native [48, 2, 8]." (same paper).
I'd be happy to share more of my thoughts; if that'd be helpful, I'd be happy to discuss my rates. Outside of that, though, I'll suggest that intuition is less helpful than experience in understanding what problems are more or less likely to have good quantum solutions.
Nah, you already showed your so-called expertise as "a trans-woman who is very good at quantum" is not the hot shit you think it is provided you went over that QRAM issue carried away by overconfidence.
As for your snarky remark on intuition, these papers by Coecke and Aerts, his thesis adviser, explains both what "my" intuition was focused on (quantum effects as perceived through Zipf distributions in linguistic data) and what was the driving mechanism behind it.
> Another finding that we will put forward, in Sect. 4, was completely unexpected. The method of attributing an energy level to a word depending on the number of appearances of the word in a text, introduces the typical ranking considered in the well-known Zipf’s law analysis of this text (Zipf 1935, 1949).
Well guess what ? I've been expecting that exact result for a decade (why would I still be tracking the progress in that field every 4 months otherwise ?) My notes linking "semantic energy levels" to word frequency date back to 2014, the observations I made in real data and that kickstarted the heavy rain of synchronicities I experienced afterwards date back to 2012. I've always known though I wasn't measuring shit – I was the one being measured and never felt like I was discovering something but was being discovered. I wanted to isolate that phenomenon and as a result (of failing to do so probably) I got isolated. There is something deeper to these subject-verb-object inversions, there is even a paper about it and I think Aerts haven't gotten wind of it, maybe with your extreme expertise you'll be able to figure it out and carry the message better than I would.
Hi, I'm a dev on Q#, and wanted to help clarify a bit how the Quantum Development Kit relates to LIQUi|〉. Microsoft's Quantum Development Kit is a successor to LIQUi|⟩ and we're excited for you to download it and begin writing Q# code. LIQUi|⟩ is focused on the efficient simulation of a quantum computer on a classical one. Q# is a language that will let you program actual quantum computers (in the future). As such, there are areas where LIQUi|⟩ can currently out-perform Q# when it is used on one of the provided simulators (which are striving to emulate the actual gates on quantum hardware). Over time, more simulators will be provided for Q# that will meet (and exceed) the efficiency of LIQUi|⟩ and at that point, we will fully retire LIQUi|⟩.
Hi, I'm one of the devs on Q#! We're excited to make this preview release of Microsoft's Quantum Development Kit available on Windows PCs. We're exploring the possibility of supporting other platforms in the future and will share more about this at a later time.
That would also break the trick of declaring something `const volatile`. While that seems like a contradiction, I've heard of it being used to force the compiler to include a symbol in the final object file. In particular, I've seen it used to make a poor-man's plugin architecture.
It's not as crazy as you think. "volatile" means that something else other than us can change the value, somehow. "const" means that we aren't allowed to write to it.
So what is that? It's any kind of input. Like memory mapped GPIO.
With that approach, each call to ``make_window`` produces a distinct type, such that isinstance couldn't be used to compare two windows. That kind of breaks the idea of a "type", I think, as being a collection of values.
"This move makes me very reluctant to work at google."
If removing barriers to getting good people interested in what you supposedly love makes you "reluctant," I think that says far more about you than about Google.
"Would my future career be limited, because women are favored in promotions and internal recruitment?"
Anyway cgranade, would you support internal career tracks for women only, to rectify the shortage of women higher up?
Do you think women should be given extra consideration for promotion at google?
No, not proof by analogy, but explanation by analogy. As Scalzi says in his follow-up, providing proof of endemic sexism is about as necessary by now as providing evidence that gravity exists when tripping over one's shoelaces. The evidence by now is clear enough that to demand evidence in every discussion is a distraction tactic, and not actually useful. Finding evidence is, because of the sheer breadth and extent of the problem, as difficult as using your favorite search engine to look up income stats, harassment at the workplace, unfair hiring practices, etc., such that demanding evidence is pretty much asking someone else to do your work for them.
(I should note that Scalzi did link to another post with more facts, over at http://www.jimchines.com/2012/05/facts-are-cool/, if you still demand that someone else go search for things online for you.)
>[B]lack males receive [prison] sentences that are approximately 10% longer
I don't see how this is related to programming but yes it is correct. The effect is even larger for men/women, so that men recieve much longer sentences for the same thing than a woman would.
>The ratio of women’s and men’s median annual earnings was 77.0
This is mostly but not fully explained by choice of career. Women in the cities of US actually earn more than men.
I don't feel like responding to more, since they are unrelated to the topic at hand, proving discrimination in the workplace keeping women out of programming.
>providing proof of endemic sexism
It depends on what you define as sexism to be sure. If you define sexism as telling a woman she looks beautiful, or showing pretty women in your presentation, then yes of course you will find things like that.
>demand evidence in every discussion is a distraction tacti
Yeah, I guess it disctracts from reading poorly thought out analogies...
>income stats
Correlation does not prove causation.
>if you still demand that someone else go search for things online for you.
Wow are being really hostile. I have made no such demands. But if you try to "prove" your point by doing analogies or by linking to irrelevant stats like incarceration of black males, I will call you out.
From my anecdotal experience women are favored over men for hiring at megacorps. I think men are still treated better at work and favored for promotions though.
Quick edit/evidence: 17% of Google technical workers are female where only 11% of CS majors at my school were women.
Krugman also backed his argument with data, notably the exact same data that Silver is disparaging by hiring people like Roger Pielke Jr. as staff writers. The data is in, and has been analyzed in a multitude of different ways: climate change is real. To ignore that is to ignore the role that expertise sometimes has to play in understanding and interpreting data.
While it's superficially true that this expertise can be used to enshrine unscientific dogmas in the trappings of science, to reject expertise period is to reject the role that knowledge can and must play in forming an understanding of our world. Data do not exist in a vacuum, but are collected in actual experiments, the complexities of which must be understood to gain insight from that data in a way that reflects reality. Not being wary of (or even intentionally misconstruing) how data is collected and represented leads to mistakes like normalizing away trends, then claiming those trends don't exist; this is precisely what Pielke Jr. did, and what Krugman was criticizing.
In a way, things like this are a part of the whole bloody point that opponents of the NSA have been making: if you put that much surveillance power into the hands of a relatively small number of humans, then they will abuse it. These sort of incidents reveal through their pettiness some of the ways in which massive surveillance invites abuse.
I agree with you, but I'd have hoped the outrageous shit happening now would be enough to break the two-party dynamic, but I'm rather doubting it, having seen that the reactions are defined by the media in an entirely two-party manner. I think if anything from the NSA leaks changes this dynamic, it will be the exodus of technology dollars from the US.
- Current devices, as well as devices likely to be built in the near- to medium-term are quite limited in the number of qubits that they implement. The current record for the most fault-tolerant qubits in a single device is 1. That's a hell of a lot better than where the field was at a couple years ago, but it's far from the huge amount of data that needs to be processed for LLM training and evaluation.
- Even if you have enough qubits to store training data, looking them up on a quantum device is still challenging due to what's sometimes called the qRAM problem. It's not trivial to make a quantum oracle that returns the data stored at a given index, and it's still an area of ongoing research to figure out how to do that.
That's part of why you see quantum algorithms being developed less for big data tasks and more for big compute tasks like chemistry. There, the program might be very large, but size of the input that has to be stored within the quantum devices and the size of the output you measure back out are both quite small, even down to a single floating-point number in some cases.
(source: I've worked in quantum computing for about twenty years now.)