I don't know how it works, but it seems very odd that serialization/reflection wouldn't work with AOT... the information you need is there in memory either way, isn't it?
It can get really tricky: using reflection you could read a string from any input and create a generic type instantiation that never happens in the source code. How would the code for that type be present in an AOT scenario?
There are also several optimizations that can be made in AOT compiled code that are not allowed when you enable unrestricted reflection or dynamically loading code. As an example, suppose an interface is implemented by just one type in your code. If you AOT compile it, the compiler can safely assume that all calls to methods on that interface actually go to that single type implementing it, thus it can replace interface dispatches with direct calls.
You can get serialization to work, but you have to use a serialization framework that is trimming compatible. For example, to use System.Text.Json, you have to opt-in the types you want to serialize.
AA sounds like a horrible company. They had a design legacy from when their logo and identity were designed by Massimo Vignelli, but they threw it away and didn't seem to have any concept of its value. Their website has terrible design and usability. When a blogger posted an article about how bad it was, one of the designers at AA reached out anonymously and said they understood his criticisms and that they were working to make the site better. AA searched their mail servers to find that employee and fired them for... caring, I guess?
I don’t know why floats aren’t included, but any float can be easily represented by an int with same bits, or a bytestring, using the struct module to convert between them, so there are clear workarounds.
A lot of people sing BB's praises but I never had a good experience with them. The client was always slow, buggy, and resource hungry, and its UI is terrible. They got shirty with me for reporting bugs when I was using a macOS beta. And finally, at some point even though nothing about my computer changed (it was a Mac Mini, what was going to change), I got a message saying some security/copy protection system had detected that my computer was "different", and I had to un-install and re-install the entire app to fix it (there apparently being no easier way to unset a flag). I uninstalled and skipped the second part.
Instead of using BB, get a Synology/Qnap/FreeNAS box to backup all your stuff locally, and back that up to another service (e.g. Glacier or Synology's own C2).
I caution the casual reader against glacier. It's not what it appears at a glance. Your files should be put into a single archive before upload otherwise you'll spend weeks waiting for AWS scripts to manage old files.
We have 23TB of images stored in S3 and I was recently looking at moving them to Backblaze to save hundreds of dollars per month. These are all individual image files, because reasons.
Then I realized that S3 Glacier and Deep Archive were even less expensive than B2. I took a bit further of a look and found that Glacier/DA files have some fairly chonky metadata that must be stored in normal S3, and for a lot of our images the metadata was larger than the image in question. So Glacier/DA would increase our storage costs. Over all it probably wasn't a money-saving situation.
The ideal use case is to bundle those up into a tar file or something and store those large files, and manage the metadata and indexing/access ourselves.
Every file uploaded is considered an immutable archive. It does not have a version history. So let's say you have 100,000 files you backed up and want to update them and don't want to pay for the storage of the old files. You need to request for a manifest of hashes for all files. This will take a few days to generate then you will be given a json file that is over a gigabyte. Next, you will write a script to delete each file one at a time, rate limited to one request per second. Have fun.
Are you maybe referring to Glacier "vaults" (the original Glacier API)? With the old Glacier Vault API you had to initiate an "inventory-retrieval" job with an SNS topic etc. It took days. Painful.
But these days you can store objects in an S3 bucket and specify the storage class as "GLACIER" for "S3 Glacier Flexible Retrieval" (or "GLACIER_IR" for S3 Glacier Immediate Retrieval or "DEEP_ARCHIVE" for S3 Glacier Deep Archive). You can use the regular S3 APIs. We haven't seen any rate limiting on this approach.
The only difference from the "online" storage classes like STANDARD, STANDARD_IA, etc is that downloading an object with GLACIER/GLACIER_IR/DEEP_ARCHIVE storage class requires first making it downloadable by calling the S3 "restore" API on it, and then waiting until it's downloadable (1-5 minutes for GLACIER_IR, 3-5 hours for GLACIER, and 5-12 hours for DEEP_ARCHIVE).
I just bought a Quantum Superloader3 last Xmas. Each LTO-8 tape (it can take 16 in 2 magazines, I use 15 + 1 cleaning tape) will hold 12 TB without compression, 30TB with, and 7 of them can back up the 100TB used on the 128TB of disk that is the house RAID array.
It takes about 2 days to make a full backup, and I can fit incrementals for the next 5 days on the batch-of-7. Then I switch to the second magazine, and do the same thing. I actually have 3 magazines, one of which I swap in and out every week, and during the before-times, I'd take that off-site to work.
I have ~30 years of data, from back when I was in college and writing CD-ROMs for backup, all on the one system. Admittedly, the major space-taking thing is the Plex library, but I wouldn't want to lose that either. It takes about 5 minutes to walk into the garage (where the server-rack is), swap magazines and I'm done - the rest is automatic.
I have vague ideas for writing a more-efficient tar designed for this specific type of setup (big disk with attached tape). The best way to do it I think is to have multiple threads reading and bzip2-compressing data, piping blobs through to a singleton tape-writer thread. Every now and then (50GB, 500GB, 1TB ?) close the device and reopen the non-rewindable device to get a record-marker on the tape, and then store the tape/record-marker/byte-offset etc. into a SQLite database on the disk. That way I'd get:
- High levels of compression without making the tape head wait for the data, which ruins the tape head. Multiple threads pooling highly-compressed data into a "record"
- fast lookup of what is where, I'm thinking a SQL LIKE query syntax for search, against the disk-based DB. No more waiting for the record to page in from the end of the tape.
- fast find on-tape, since you'd know to just do the equivalent of 'mt fsf N' before you actually have to start reading data
Right now, tar is good enough. One of these days when I get time, I'll write 'bar' (Backup And Restore :)
Too loud for an office. Which is a shame because in these pandemic times, I repurposed the shed at the bottom of the garden as an office (insulated, add power and a/c) and I wanted it there.
It's not noisy when writing to tape, but the mechanism is noisy when a tape is being loaded, the magazine is being shuffled to get the right tape, etc. It's the mechanical parts rather than the tape drive itself that's too loud, especially with Webex conferencing being a part of the day now.
So I have it set up in the garage, in the server-rack. I was worried about temperatures in the Summer, so I bought a 100W solar panel, an attic fan, and linked them up, positioning the fan above the rack. That fan shifts so much air that the in-rack fans (with temperature monitoring) didn't get above 85 all summer, which is pretty amazing for the Bay Area. The tape deck seems to be fine in that sort of temperature, and yes I do do the occasional 'tar tvf' to check the data is readable :)
Yes. I'd forgotten about that. There is a reasonably-above-ambient-noise fan that is running constantly. Another reason it was banished to the server-rack in the garage.
I got mine at Backupworks.com[1] - they seem to have a perpetual sale on for pricier items like this. They gave me a discount on a batch of tapes bought at the same time and threw in some barcodes and a cleaner tape as well.
You need to differentiate between BB Personal Backup and BB B2 service which is more like something you suggested. But these days I just use rsync.net + Wasabi + Kopia + rclone.
Do you understand that popping up like Beetlejuice every time someone mentions rsync pricing and being snarky about an unpublished discount is not a good look?
Particularly given you never address the commenter's point, which is that your pricing is pretty expensive?
"Price high, low volume" is certainly a valid pricing strategy, but you have no right to be snarky when people say you're priced high.
The fact that people are consistently describing you as overpried means your marketing really isn't showing the corresponding value to them.
Not sure if they're overpriced when they also have borg discount and rsync.net is one of the only service that I can borg or "zfs send" directly as well as the fact that they've been running nearly 20 years? to know they don't disappear next year makes it rather feel not overpriced.
I have 3.5TB backed up to Backblaze for $5 a month. I haven't found any other online backup option that provides anywhere near that cost.
I use a RAID 1 to handle drive failure and also keep local backups on a NAS. BB is my third layer of backup. I've never run into issues with BB backups so I'm happy for what I get for the price.
S3 Glacier Deep Archive, $0.00099 per GB per month.
I have a ZFS based NAS. And periodically do a incremental backup (zfs send) of the entire dataset, encrypt it gpg and pipe it straight up to S3 deep archive. Works like a charm.
The catch with S3 deep archive is if you want to get the data back... It's reliable, but you will pay quite a bit more. So as a last resort backup, it's perfect.
The very first time you do it, you will need to do a full backup (ie. without the `-i <...>` option). Afterwards, subsequent backups can be done with the -i, so only the incremental difference will be backed up.
I have a path/naming scheme for the .zfs.gpg files on s3 which include the snapshot from/to names. This allows to determine what the latest backed up snapshot name is (so the next one can be incremental against that). And also use when backing up, since the order or restore matters.
Ah gotcha, I haven't done full restore of my main dataset.
I've only verified with a smaller test dataset to validate the workflow on s3 deep archive (retrieval is $0.02/GB). I've done full backup/restore with the zfs send/gpg/recv workflow successfully (to a non aws s3 destination), and used s3 for quite a long time for work and personal without issue, so personally I have high confidence in the entire workflow.
> get a Synology/Qnap/FreeNAS box to backup all your stuff locally, and back that up to another service (e.g. Glacier or Synology's own C2).
I'm not a big fan of backing-up a back-up and opted for a Time Machine backup to a local nas and in parallel, an off-site backup to B2 with Arq on my macs.
I have a Synology NAS currently and use their HyperBackup hosted tier. Maxes out at 1TB sadly, not even enough for my laptop backup. C2 on the other hand is well priced for bulk storage but does not have the “free” deduplication.
The hosted version I'm thinking of is their Synology C2 personal it seems.
Currently I use it to back up my lossless music collection and nothing else.
Backblaze B2 was on the table too, however I think my asymetric internet connection is my biggest issue right now. Only 40-50Mbps upload won't do much for backing up multiple TBs of data. May need to consider pre-seeding drive option if I can justify the cost.
Other solution was a separate, lower spec Synology I can pre-seed and send to a friend's house who has a homelab.
