Huge fan of all the work you do. Wanted to understand something fundamental and whom better to ask than you: Whats so special about the transformer architecture that its able to predict the next token so beautifully understanding all the intricate previous token relationships? I understand Attention but what so special about this architecture that no other architectures are able to "attend" appropriately to previous tokens? Being a CS guy, its really hard for me to fathom that we have not yet created another architecture which can perform similarly.
Transformers have quadratic computational complexity in sequence length, i.e. O(N^2) where N is the sequence length. RNNs, Linformer, Mamba, etc. have linear or quasi-linear computational complexity in sequence length, which often bottlenecks information movement across tokens.
In theory, if you grew the RNN's state quadratically vs sequence length, you could likely achieve comparable performance to transformers, but it would likely be less efficient than transformers.
krunvm uses libkrun (https://github.com/containers/libkrun) for executing the VM, and while the later is also based in rust-vmm and shares some code with Firecracker and Cloud-Hypervisor, it's specialized in the process isolation use case. This means it implements a different set of devices (most notably, virtio-fs instead of virtio-blk, and virtio-vsock+TSI (Transparent Socket Impersonation) instead of virtio-net), and it takes the form of a dynamic library instead of a final binary.
In fact, the networking limitations are caused by this use of virtio-vsock+TSI. TSI (WIP implementation here: https://github.com/containers/libkrunfw/blob/main/patches/00...) is an experimental mechanism that provides inbound and outbound networking capabilities to the guest, with zero-configuration and minimal footprint, by transparently replacing user-space AF_INET sockets with AF_TSI, that have both an AF_INET and AF_VSOCK personality.
TSI has the additional advantage that, for the host side, all connections appear to come and go to the process acting as a VMM (in this case, krunvm, as it links directly with libkrun), which makes it very container-friendly in a way that even side-cars (such as Istio) work out-of-the-box.
Also, firecracker did not merge virtio-fs [1]. So I guess it is a scope and priority decision:
> To not trade off our core use case simplifying assumptions, we'll need something like a way to build different Firecracker "specialized variants" or alternatively a plugin system. This is something worth discussing for 2021. Again this is not a guarantee, but virtio-fs could fit in such a scheme, especially if it's a rust-vmm crate we can easily import over.
Yes, totally possible. You can use UNION ALL to merge them together. We are also working on globbing support for the Parquet reader that should be released next week [1].
From the white paper: "If additional shadow headers are used, their suggested locations are (where possible) at the sectors starting at offsets 128MiB, 128GiB, and 128TiB"
Is such repetition of header (in a supposedly random-wiped disk) not suggestive of existence of a PUREE partition? This can be figured by scanning the entire disk.
Don't? I'm currently working on a Golang web app with a small team and we're lucky enough to be each working on separate modules that fulfill a predefined specification we drafted prior to starting a line of code. As long as each module fulfills the requirements we drafted, we don't care about each others dev environments, because go creates a binary.
Also: `go fmt` is incredible (and built in to literally every Golang environment). We used to work primarily in Node.js but since making the switch last year to Golang we've noticed an increase in code-review productivity. Everyone's code looks the same so we don't have to spend time worrying about dev environments.
I would recommend to look at ProtoMail.com.
They provide a lot of transparency, even open-sourced there protocol.
If you want Users to trust their data, your Server, you'll have to provide more than a 'good encryption'.
We will soon add more pages explaining how we are doing the encryption. Few details are already mentioned in the homepage as of now. Essentially we take every 64 kb of your file and encrypt it with AES-256 encryption algorithm with GCM authentication using your VyomKey as the encryption key. We haven't rolled out any new encryption algo. This is just a start, will add more pages as we go forward. Our main USP is that we use an encryption key of user's own choice - the VyomKey - to encrypt the files which is not there in any other competing product.
Huge fan of all the work you do. Wanted to understand something fundamental and whom better to ask than you: Whats so special about the transformer architecture that its able to predict the next token so beautifully understanding all the intricate previous token relationships? I understand Attention but what so special about this architecture that no other architectures are able to "attend" appropriately to previous tokens? Being a CS guy, its really hard for me to fathom that we have not yet created another architecture which can perform similarly.