What LLM are you currently using? And how do you prevent against hallucinations/drift when generating content of this length? Do you start by asking the model to generate an outline of a book, and then have it expand on each chapter with more detail? Awesome project
Currently using GTP-4o combined with Perplexity API context infusion for real-time knowledge (this also reduces hallucination for the most part). And yes, starting with outline and then writing chapter by chapter while doing real time research. After initial completion there’s another editing round to make everything more coherent.
I still use a 2014 macbook air as my home server. It was my daily driver for 7 years. No complaints, still works perfectly. Haven't been careful with it either.
They're grown from stem cells. Neuro-ethics (especially the ethics around using brain organoids) is an entire field of research. Although brain organoids exhibit neural connections and electrical activity, they have so far failed to form even basic synaptic circuits — without which consciousness is probably impossible. But of course we should take every precaution to ensure that they DON'T form consciousness. But that's still a ways away.
What's the harm if they do form consciousness? Is the concern they will develop a willpower and work towards freeing themselves, or is it just a "don't enslave sentient beings" thing?
Are there any good resources for understanding models like this? Specifically a "protein language model". I have a basic grasp on how LLMs tokenize and encode natural language, but what does a protein language actually look like? An LLM can produce results that look correct but are actually incorrect, how are proteins produced by this model validated? Are the outputs run through some other software to determine whether the proteins are valid?
Proteins are linear molecules consisting of sequences of (mostly) 20 amino acids. You can see the list of amino acids here: https://en.wikipedia.org/wiki/Amino_acid#Table_of_standard_a.... There is a standard encoding of amino acids using single letters, A for alanine, etc. Earlier versions of ESM (I haven't read the ESM3 paper yet) uses one token per amino acid, plus a few control tokens (beginning of sequence, end of sequence, class token, mask, etc.) Earlier versions of ESM were BERT-style models focused on understanding, not GPT-style generative models.
Agreed, would be interested if someone with more knowledge could comment.
My layman's understanding of LLMs is that they are essentially "fancy autocomplete". That is, you take a whole corpus of text, then train the model to determine the statistical relationships between those words (more accurately, tokens), so that given a list of tokens of length N, the LLM will find the next most likely token for N + 1, and then to generate whole sentences/paragraphs, you just recursively repeat this process.
I certainly understand encoding proteins as just a linear sequence of tokens representing their amino acids, but how does that then map to a human-language description of the function of those proteins?
Most protein language models are not able to understand human-language descriptions of proteins. Mostly they just predict the next amino acid in a sequence and sometimes they can understand certain structured metadata tags.
Can they understand the functional impact of different protein chains, or are they just predicting what amino acid would come next based on the training set with no concern for how the protein would function?
The way you would use a protein language model is different from how you would use a regular LLM like chatgpt. Normally, you aren't looking for one correct answer to your query but rather you would like thousands of ideas to try out in the lab. Biologists have techniques for trying out thousands or tens of thousands of proteins in a lab and filtering it down to a single candidate thats the best solution to whatever they are trying to achieve.
Get a sleep study done. Lofta will ship you an at-home test. Sleep apnea will leave you feeling unrefreshed no matter how much sleep you get, and it will only get worse without a CPAP or other aid.
I would love to see a comparison of the average amount of trash (especially plastic waste) the average American homeowner produces vs the average European homeowner. I bet the difference is astonishing.
But probably everything-consumption is really high, with a toxic transportation system (sand for roads), land use, electronics use (rare earths), etc. Fucking up the planet is the easiest way to increase GDP.
$60 + lost income for a student who works outside school hours and also has to take the test outside school hours + transportation which could be over an hour in rural areas and you can see how this could quickly add up, especially for a teenager who may be smart but may not see the proper cost/benefit of skipping the SAT
All this seems like a small investment compared to properly preparing for the test. An average student would probably want to study at least 10 hours if they care about their test score.
> especially for a teenager who may be smart but may not see the proper cost/benefit of skipping the SAT
The SAT is for students who want to attend college. If $60 and a two hour round trip is going to tilt the scale on whether you spend 4 years in college, then from the college's point of view, you may not be the type of applicant they are looking for.
You are looking for corner cases within corner cases at this point. I know it's uncool to assign the responsibility of children to their parents these days... but if you subscribe to the idea that every system in place has to cater to every possible living situation, you'll just end up with one that doesn't work well for anyone, as evidenced by this post.
Absolutely agree. If these tests are not offered in the school during normal school hours, students should receive compensation for taking the test. It's bonkers to me that students have to pay to take it. Travel should also be provided if the test cannot be offered in the student's school. I think that would be a much fairer way to close the equity gap for standardized testing, rather than remove one of the few ways a student from a low-income background can prove their academic worth.
Not a dumb question, I’m assuming it’s been treated with flame-retardant chemicals, but I’m surprised that it’s fireproof-ness wasn’t addressed in the article.
Since they are the tallest thing around, wind turbine blades get struck by lightning regularly.
Fiberglas is an insulator. Aluminum is a good conductor. But carbon fiber is a resistor. Carbon fiber blades have to have embedded conductors to prevent from being blown up by lightning strikes. So there's a layer of aluminum foil or similar to provide a low-resistance discharge path.
Same problem as carbon-fiber aircraft, with similar solutions.
I would strongly suggest reading “Time Loops” by Eric Wargo if you’re interested in investigating this feeling more. There are many (documented!) instances of people “remembering” and dreaming about real events in their lives that haven’t yet happened. But of course, the scientific evidence for precognition is pretty flimsy. If it’s real, it seems to have a very small/rare effect.
The reason the scientific evidence is flimsy is that this can almost entirely be attributed to selection bias: it feels important when you make predictions that are correct, it doesn't feel noteworthy when you make ones that are wrong - you'll probably even forget about them by the time they turn out to have been wrong.
Plus our brains are pretty good at predicting what will happen based on things we've seen before and most people have intrusive thoughts (though these are usually based on actions rather than just things happening, e.g. throwing a baby out of a window or jumping in front of an oncoming train). Our memories are also extremely malleable so if a prediction didn't match exactly our brains will happily adjust the details to find a match or we'll ignore details that don't match because they don't seem as important as the ones that do.
It's spooky but the real wonder here is that our brains work somewhat reliably most of the time to begin with. Even our vision is a complete mess - ranging from the physical blind spots in our eyes to various forms of "blindness" (e.g. change blindness) and the work our brains have to do in order to compensate for saccades (i.e. why you don't see motion blur every time your eyes move).
There have been many occasions where I've been somewhere I know for a fact I haven't been before but can vividly "remember" being there, just as a still frame of what I was looking at in that moment.
My theory is that the brain is temporarily using a lossy delivery protocol - you're receiving short-term memory packets out of sequence and its causing you to feel like you're "remembering" the present.
There is no basis for my theory and it's fantastical horseshit.
