Attention Viz

[djoldman]

http://attentionviz.com/

[GaggiX]

Did the page load for someone?

In the meanwhile I have found the paper page interesting: https://catherinesyeh.github.io/attn-docs/

[banga → GaggiX]

The UI tries connecting to http://18.219.70.154:8500/getMatrixData/vit-nat for the data, but that target appears to be unavailable.

[ape4 → GaggiX]

There are probably thousands of us trying to load it then bailing part way thru.

[swyx → ape4]

maybe its open source and some of us can rehost it?

[swyx → GaggiX]

it just loaded for me after 1 failed attempt

[Lio]

The site is not responding for me. I'm guessing the page is temporarily hugged to death.

In the meantime, I originally thought this might be linked to Viz magazine[1][2]. Spoilers, it's not.

Warning: Viz deals with a very specific sort of British humour.

1. https://viz.co.uk/

2. https://en.wikipedia.org/wiki/Viz_(comics)

[kordlessagain]

I've been working on a project I've called DocGPT: https://github.com/featurebasedb/DocGPT. This is a work in progress and I'll be putting together a release next week, once I've got the instructions for running it done.

It was my assumption that GPT-4 could be used for keyterm and question extraction from the fragments of a document and perhaps used to build a semantic graph that could be leveraged for rapid prompt assembly, augmenting the vector searches that are normally done. The idea here is to set "attention flags" on the document during discussion.

In this example, I indexed the arxiv PDF for the paper: https://arxiv.org/abs/2305.03210.

Here are a few example keyterms it extracted, along with the count of fragments containing the terms:

  intermediate artifacts of self-attention 1
  attention patterns 3
  key vectors 5
  gpt-2 7
  bert 15

Here's are some questions it asked and answered, using keyterms in each fragment to identify relevant document fragments for assembly (in addition to using nearest neighbor vectors):

system> How can attention head comparison and model pruning be used to identify anomalies in transformer behaviour?

bot> Attention head comparison and model pruning can be used to identify irregularities and potential behavioural issues with transformers through attention pattern exploration, which can provide insights into why large language models fail at reasoning tasks or math. Comparing heads could allow to find parts of the model that are actually useful, and tracking attention flow throughout the model could help with hypothesis testing and visualizing induction head pairs.

system> How does Attention Viz help to visualize multiple datasets from different NLP tasks?

bot> AttentionViz can identify consistent shapes and patterns across different NLP tasks and can also identify anomalies and unexpected behavior, as well as facilitate attention pattern comparisons.

system> What are the mean correlations between query-key distances and dot products for BERT and GPT on Wiki-Auto data?

bot> -0.938 for BERT and -0.792 for GPT

system> What technologies are used in the Attention Viz prototype?

bot> The AttentionViz prototype uses a Python/Flask backend, a Vue/Typescript frontend, a DeepLabv3 segmentation model and Deck.gl to visualize the corresponding query-key joint embeddings.

[qingcharles → kordlessagain]

This sounds like exactly what I need. I tried ChatPDF last night and paid the Plus fee, but had terrible results, sadly.

I have about 9,000,000 PDFs I need summaries of and I can't find a good AI solution as yet...

[kordlessagain → qingcharles]

This sound impressive and a good challenge. Let me know if I can help. We’d need to do some map reducing on it.

[mrwnmonm]

What does attention visualization mean?

[Nzen]

tl;dr a 'parse tree' for how their (transformer based) LLM weights the import of the tokens in a given prompt. The authors noticed patterns in the visualization that they felt imitated relationships that we would regard as related.

That's as much as I can summarize, given my ignorance.

[kordlessagain → Nzen]

Use the force!

user-aRxb[attentionviz.pdf]> summarize the document's contents

bot> Querying GPT...

bot> This paper describes AttentionViz, a tool designed to provide a global view of Transformer Attention that can be used to identify sentence view, clusters of search results, semantic behavior, and fine-grained attention patterns; experts have suggested additional interaction modes and global search patterns to be used to quickly scan for and compare attention trends.