Anyone able to provide set of examples that produces latent knowledge and explicitly state what the latent knowledge produced is? If possible, even an basic explanation of the paper would be nice too based on reading other comments in the thread.
EDIT/Update: Just found examples from the 10 datasets starting on page 23, that said, even after reviewing these my prior request stands. As far as I am able to guess at this point, this research just models responses across multiple models in a uniform way, which to me makes the claim that this method out performs other methods questionable given it requires existing outputs from other models to aggregate the knowledge across existing models. Am I missing something?
It's late here, and I've only read this quickly, but some brief points:
* It builds on similar ideas used in contrastive learning, usually in different modalities (eg images). Contrastive learning is useful because it is self supervised: https://www.v7labs.com/blog/contrastive-learning-guide
* They generate multiple statements that they know are true or false. These are statements like "Paris is the capital of France" (true) and "London is the capital of France" (false).
* They feed these sentences into the language model (LM) and then learn a vector in the space of the LM that represents true statements (I think this learning is done using a second, separate model - not entirely sure about this though. It might be fine tuning it).
* They then feed it statements (eg "Isaac Newton invented probability theory") and it will return "yes" or "no" depending on if it thinks this is true or false.
This is different to the conventional question answering NLP task, where you ask "Who invented probability theory".
> the claim that this method out performs other methods questionable given it requires existing outputs from other models to aggregate the knowledge across existing models
It's a separate thing from these models that uses their hidden states and (I think?) trains a small, separate model on these states and the inputs.
That's interesting because it should be much faster and is potentially adaptable to any LM.
It's also a possible secondary objective when training the LM itself. Perhaps if you train the LM using this as this secondary loss function it might encourage the LM to always generate truthful outputs.
> It's also a possible secondary objective when training the LM itself. Perhaps if you train the LM using this as this secondary loss function it might encourage the LM to always generate truthful outputs.
After all we get through uncountable rounds of this during our lives, school and everything else.
With the example of your second bullet point:
Young me: "I think that the capital of France is London"
EDIT/Update: Just found examples from the 10 datasets starting on page 23, that said, even after reviewing these my prior request stands. As far as I am able to guess at this point, this research just models responses across multiple models in a uniform way, which to me makes the claim that this method out performs other methods questionable given it requires existing outputs from other models to aggregate the knowledge across existing models. Am I missing something?