Ubuntu's backup thing (duplicity?) does something like this too. I can right-click in a directory and restore deleted files, or on a file and view previous versions.
One suggestion for the author of that webpage, and a common complaint I have with open source software websites in general: tell us what your software does. I haven't used a mac, I don't know what features time machine has in any sort of detail so telling me it's a clone is pretty useless. Give a one-or-two sentence saying what it does, and then some bullet points of key features.
There's more description of the name than there is of the functionality.
Duplicity works by making a full backup and then series of incremental backups based on it. Eventually you can make another full backup and new series, but you cannot delete full or incremental backup without rendering whole series corrupt. In Time Machine (and - I presume - this software), if you delete a backup checkpoint for a specific date you lose only file changes captured by that checkpoint, but state of the backup at the checkpoints before and after remain intact.
I believe the correct name for what time-machine does is differential backups.
Instead of doing a full backup and then having a chain of incremental backups, each differential backup stores all the differences from the full backup, so they are much larger but it allows you to delete previous differential backups.
One of my guiding principles in UX is pretty much this. Be explicit about describing the thing that you are talking about. Then make it even more explicit.
In the past, duplicity would not save dot files or dot directories, e.g. .config/, .emacs, .local/, so many settings would be lost or not saved. No idea if they addressed it meanwhile, but back then it was a show stopper for me. Lost some of my steam savegames :-p
What I would like is something similar but for my local zfs/btrfs disks. Copy snapshots to an external drive, but the issue I have more often than not would be deleted/overwrittem files rather than damaged disks which regular snapshots would resolve. I don't always have access to my NAS.
What i really miss on Linux, or for backups in general:
Make it super easy.
I just want to open the software and tell hey here is my network drive/external drive. Please start the backup of all my installed apps, all my configurations, all my music and develop files.
Also for restoring, just make it easy. Can somebody point me to a good solution? The problem is Linux is too diverse. Either you use basic tools like rsync which everybody can use, or you get super specific with automated btrfs backups onto a specific btrfs drive with deduplication and backup snapshots. Sounds super useful. But not easy as well. Also a gui would be nice, or at least a super in depth tutorial walking you through backing up your apps, your configs, your ssh keys. whatevs.
I lazily set up Duplicity one day (the tool that comes with Ubuntu) and it's been working ever since. Has support for retrieving backups, allows you to recover individual files from the file explorer (at least on Ubuntu, that is) and has a one-click full backup restore.
It comes with a GUI in which you tell it what to backup what to ignore, where to backup to and when to run the backup schedule. It'll also ask for a backup password to secure the backup and will ask you to enter it once in a while to make sure you don't forget it. It also makes full backups instead of partial ones to ensure that delta backups don't become corrupt.
It's mostly a python wrapper around rsync but it does all the annoying stuff for you.
The thing is the only real backup solution which worked for me was clonezilla. Offline backup of a complete drive. But it is too much work. Its offline, it takes time etc. But at least i have a known working state which i can restore onto and have everything. I would need a live clonezilla backup onto a network drive and a bootloader, which has functionality to restore that backup.
I really like duplicati. It works in Linux, Windows and MacOS. It has good support for tons of back ends for storage (nfs, cifs, dropbox, s3, ssh, etc). The software is FOSS and it has a design that is well suited to cloud storage: it makes blocks of changes that are encrypted. As long as you have the password, it can use just those blocks to decrypt and restore.
It does automated restore tests. Setup is pretty easy. You install it, navigate to the internal web page and follow the wizard. It has sane defaults but you can change it to skip stuff you don't want backed up.
Duplicity is OK but it tends to be pretty CPU heavy when it runs and you can't control when it runs with much granularity. I used to use and like CrashPlan but its current pricing model doesn't work well for my home environment.Granted, my backups weren't huge so the memory usage of CrashPlan was OK for me.
The GUI is nice and intuitive, and it supports both simple rsync snapshots and more complicated btrfs drives. It took me about five minutes to set it up to automatically back up once a day / week / month.
My personal setup excludes the home directory and saves backups once a week to a `/timeshift` folder, but you can easily set it up to write to an external drive, include your home folder, or back up manually. Somebody even wrote a script to take a backup before a system update with pacman hooks.
But really if you want to make it dead simple you can't go wrong with tarsnap (https://www.tarsnap.com/). It's written and maintained by one of the biggest names in the crypto community, the guy who has spawned scrypt, among other things.
> But really if you want to make it dead simple you can't go wrong with tarsnap
I'd call this everything but dead simple. Maybe for the average software engineer it's okay but I don't I could tell anyone in my family to set this up themselves and then monitor that it's doing it's thing correctly.
TimeMachine is two clicks in self explaining steps (Plug in empty external drive, OS asks if you want to use it as a backup source, you click "yes" and it starts doing backups every hour). That's probably what GP meant with "Make it super easy." not this: https://www.tarsnap.com/gettingstarted.html
> But really if you want to make it dead simple you can't go wrong with tarsnap (https://www.tarsnap.com/). It's written and maintained by one of the biggest names in the crypto community, the guy who has spawned scrypt, among other things.
Well, patio11 said several years ago that tarsnap should charge a lot more. But even at its current prices, the service is way too expensive for personal backups that run into hundreds of GBs. Personal data may not be significantly amenable to deduplication. So almost all the data will be counted for billing. It may be a good solution for tiny backups that run into a few MBs and/or have a lot of duplicated data across the filesystem that's being backed up.
Can also recommend BackInTime, which is just a GUI for the rsync command-line app - like all good Unix utilities should be. Though BiT is Python 2 I believe, and I don't know if it did make it into Ubuntu 20 LTS yet (probably yes, and there's nothing to worry).
I'm using BackInTime in three workstations for almost 3 years. Never let me down once.
I love how it stores the configuration in the backup folder. You just mount your backups, fire up BiT in a new machine, show the foler, and viola!
Addendum: BiT is python3. Debian Testing removed all python2 packages from mandatory installment 3 days ago. I have no python2 runtime on my system, yet BiT runs happily.
BackInTime is an excellent alternative to TimeMachine! Searching old backups is literally just browsing directories. I've used it for years until Ubuntu started nagging to use the included backup utility.
Once you make the first snapshot, BackInTime is much faster that the recommended backup. Where the native backup shines is encryption and file splitting: you could backup to a cloud drive and not worry about strangers looking at or extracting metadata from your personal files.
Time Machine is one of the best features of macOS in my opinion. Apparently your mileage may vary, but I’ve never restored a backup to a new computer with such ease as I did when I swapped a MacBook Air for a MacBook Pro last year. I’ll be saving this and giving it a try on my Linux boxes.
Could you elaborate on how Time Machine doesn't work? It's been pretty trouble free for me; other than taking a ridiculously long time for the initial backup with no good indication of progress.
OK. I have done that, several times [x], and didn't run into any problems.
I do wonder, though, what kind of integrity checks do Time Machine backups have. I've always treated TM as the first line of defence and, as such, it has worked well for me. However, in addition to TM, I have other offsite backups in the cloud for disaster recovery.
x) I haven't had my macs serviced several times, but I have restored from backups a handful of times due to, ahem, accidentally rendering the system unbootable a couple of times.
TimeMachine underneath is pretty much a fancier rbackup. There are no integrity checks, but there are checks for motherboard serial number, as that's how backups are matched to systems.
That said, a friend of mine had been complaining that TimeMachine backup didn't backup anything of his Homebrew-installed stuff...
This is actually how I have TM set up. I ignore anything not in ~/Documents (or any other location with personal files). The other stuff is so easily replaceable that I'm not concerned if I lose it.
If you use network storage for backups (AFP server) backup can get corrupted if the network goes down, and in most cases I have seen it is unable to recover and continue. You just have to delete the whole backup and start from scratch. This is why I use Vorta (Borg GUI) on macOS instead.
OMFG...This is one reason why I just started using freefilesync and just mirror my data instead of my whole drive. That way I just gotta have the hard drive and I'm set. Although it sucks cause I have to redownload every piece of software again.
Time Machine at least used to have a ton of reliability problems.
I'm hoping that it's improved recently which I suspect would happen especially after Apple switched to AppleFS. Time Machine on HFS+ was basically hacks extending hacks extending hacks extending HFS.
I strongly agree, while I'd rather run Linux, this is the reason I can't get off mac. I have restored backups at least 5 or 6 times over the years, including last week. Once, I had to jack around with wiping a fresh Catalina before the restore worked, because of APFS/HFS issues, which the restoration UI complained about. But it has always worked when I need it. Post-restore, I'm exactly back where I was. With hundreds of gigabytes of geospatial data, and a decade of software projects, I can't start from scratch without losing weeks of time.
I believe the Time Machine is a white elephant, a relic.
It ought to be able to do these things as good as (if not better) apps like restic, borg, backblaze etc : dedup, compression, unlimited versioning (at file level), integrity checks, granular search/restore, possibility to set a cloud destination (at least iCloud).
The distinguishing feature of Time Machine, and one which this project seeks to emulate, is its use of file navigation on the z axis.
Given the requirements of this particular task, this is the perfect choice. 3D navigation of the OS has been tried many times (Microsoft Bob anyone?), but to my mind, this is the only time it has really worked.
IMHO, 3D navigation in Time Machine is more of a UI gimmick than actually necessary.
I think I would prefer a calendar view at the top, with dates/times that have snapshots highlighted, and a file manager view at the bottom. Kind of like Wayback Machine.
The Time Machine mental model is more like "I want my computer to be exactly as it was at 2pm on March 17, 2020."
Your suggestion is closer to "I want to see the revision history of this particular file, never mind the rest."
For me, the distinguishing feature of Time Machine is how wonderfully primitive it is. You can mount a backup volume and just browse the hardlinks. I've written simple Ruby scripts to dig out some information. I couldn't have done that if it were a binary black box, like everything else is nowadays.
To me, the distinguishing feature of Time Machine is that it reliably works. It worked reliably when I first bought a Mac 12 years ago, it works reliably today. I'll put up with the fact that it's slow and naive (file and hardlink based), and the UI gimmicks are kind of unimportant to me. I care about the recovery.
I literally just today had to kiss 2 months of some folders goodbye because TimeMachine utterly randomly decided to stop backing them up. First it decided to stop backing up the home folder; happened to catch that before reformatting. When later I reformatted.... some folders in ~/Documents were empty. Ironically important ones (“archive - important” :( ). I sleuthed through logs, exception rules, everything: no rhyme or reason, no explanation. It just didn’t back them up.
Pure luck that the last reinstall was 2 months, and that version did back it all up. Lesson learned: don’t trust Time Machine. It’s a nice to have.
I'm glad it works for you, but I abandoned it due to unreliability. My experience has been that it works fine for some period of time and then the sparsebundle will randomly break. MacOS claims it is corrupted, I have to jump through hoops to make it work again.
I will note that I'm doing it over a network mount, not a local drive, and this is probably the source of the problem, but I never could resolve it. So I'm just back to rsync for specific directories. Which is fine, when this Mac dies, I'm moving to Linux.
It was never real good about backing up over a network except with hardware specifically designed to support that, like network routers that support Time Machine drives (or Apple's own Time Capsule).
I don't know that I'd say my own experience with Time Machine is perfect, but it's generally done what I needed to do, and has actually saved my bacon a few times by letting me restore individual files back to specific versions easily -- something I'm not sure an rsync-based solution would really provide (e.g., versioning for everything).
I‘ve said that many times and here I am, still working on a 2012 MBA refusing to die. But its performance is still fine anyways. Single core performance seems to only have about doubled since back then, so its not that much of a dealbreaker
TimeMachine and reliable? I manage a TimeMachine (Mac)+Acronis (Win) deployment at work (storage: QNAP TS 1635) and it's a freaking mess.
Acronis: fast, reliable, tracking UI that shows status for admins, can be centrally deployed while allowing users to set their password individually, supports different backup schedules/retention periods and quotas
TimeMachine: dog slow (unusably so on Wifi), no central tracking, once a week a user has to fully wipe their backup because it somehow got corrupt, fully manual deployment necessary as there is no way to implement encrypted timemachine backups from a script, no way of individual quotas or restricting backup history...
I know whole backups were highly unreliable with TimeMachine, and this was a known problem.
I had to use it twice to setup a new replacement laptop with the same data as my old one. The first time it went smoothly, but the second time it failed badly.
The mechanism of rsync + hardlinks is similar to BackupPC[1][2], which is an 18 year old open source backup tool, written in Perl, with a web UI. It supports backing up Mac, Windows, and Unix-like OSs, and is primarily for centrally backing up multiple computers. BackupPC's UI is obviously not as slick as this, but it does allow restoring individual files from a given date. If I remember correctly from 10+ year ago, their tar over SSH transport option performed better than rsync when used on a fast network.
For folks that want to back up data on remote servers, BackupPC is one of very few comprehensive backup systems I've found that can run on a server hidden in a closet somewhere and connect out to the remotes to retrieve data. Most backup systems run on the remotes and have the remotes connect into the backup server in some fashion, which I find kinda horrifying.
I've used BackupPC for a long time and I totally love it, though I get why it's never been the most popular option. The initial setup will take an experienced sysadmin a couple of hours if they're doing it for the first time.
This is pretty straightforward to do with standard rsync. You just setup a ssh tunnel first and have it use that. That's how I backup my VPS from my home fileserver.
I really liked BackupPC, but found the storage requirements to be pretty staggering for ~100 machines. IIRC (it's been a decade), large files that were appended to (log files, Zope DBs, pretty much any database), would get fully copied and stored for every backup copy saved. I also recall backup performance to be pretty bad, we had a number of systems that couldn't backup daily because they took >24 hours. We were adding new BackupPC servers constantly.
I ended up building a backup system that was a thin layer and used SSH+rsync+ZFS for all the heavy lifting. Eventually added a web-based GUI. Went from 10 backup servers to 1 because of ZFS snapshots.
Generally, I liked borg, but I did run into a situation a few years ago where borg would fail to recover parts of my backups because of some unicode error. I didn't knowingly have any unicode filenames in my home directory, but something annoyed it to the point that it would bomb out during recovery. I tried for a few hours using different mechanisms to do the recovery, but couldn't find any way to resolve it.
So, as always with backups, make sure you do regular test recoveries.
It delegates all the actual file operations to rsync. In practice, it looks like it does multiple scans each hour - glob on all excluded folders, then running rsync with a big list of excludes for each configured folder.
There no clever modification monitoring. I'd keep this away from folders with many thousands of files.
> Although neither the webpage nor README explains how it works, I'm guessing it misses a vital Time Machine feature, which is FSLogger.
I don't know what FSLogger is, and you don't describe the functionality, and you say that you guess that this new free software suite doesn't provide it.
Is it something like inotify on GNU/Linux?
I do constant backups of 500GB of data via syncthing from my main machines (desktop, laptop) into my archiver, which then uses borgbackup to run an archive / snapshot periodically. Syncthing takes at most a minute to pick up on changes via inotify.
Is this the kind of OSX-only killer feature that you're hinting at?
FSLogger is like inotify but for the whole disk. Inotify is just for files and directories; it doesn't scale to millions of directories. It also would require a full rescan on every boot because installing the watches will take so long you will miss changes.
I "guess" because no backup program for Linux supports watching whole filesystems for events. There are ways it could happen with recent kernels, but they are poorly documented and not designed for backups. Nobody has done it so the performance impact is also unclear.
AFAIK, inotify only works on a single directory. You can set it up to add an inotify to newly created directories, but you need to hook into every directory in the filesystem. Unless this has changed more recently, you can't just set up an inotify on "tell me when anything has changed on this filesystem". Some google searches do not indicate that this has changed.
I had initially thought that it was just a MacOS X app clone but however since trying it out I have found that it is actually better than the app was trying to replicate. I'm gonna port this to gentoo now, see you guys in ten years.
> The name comes from anacronopete ("who flies through time"), which is a time machine featured in the novel from Enrique Gaspar y Rimbaud, and published in 1887 (eight years before than H.G. Wells'Time Machine).
"Pete" seems to work as slang, and it kinda makes sense to me reading the UD explanation, I was just confused by the Chrono part in the original comment.
Can someone explain to me why Time Machine is so great? I've used it, but I've just never found it worthwhile in comparison. For me it was very slow (but worked in the background which was nice), only mimiced my drive based off of whats currently on it and if I deleted it in the past.
I guess because I have several backups that I rotate, that essentially is my time machine that only goes as far back as my last backup for that drive.
I think the killer feature of time machine is that it's dead simple and intuitive for non technical users to do regular full system backups.
You plug in an external HD, open time machine, flip a big switch that goes from "Time Machine is Off" to "Time Machine is On" and the software does the rest.
Full backups, incremental backups, reminders to connect your drive when it hasn't been backed up for a while, and a visual interface for selecting previous versions to restore to.
My problem with it is that I still need another mac to use it. In the event of an emergency (IE theft or disaster), the last thing I want to hinder me from accessing my backup is an operating system. Especially a high cost and not as commonly used in public spaces operating system.
i've been using restic daily to a local store (with compression and encryption turned on), and then once restic has finished its daily run then i rclone to backblaze. i keep one snapshot per day for a week, 1 weekly snapshot for a month, and 6 monthly snapshots. i pay about £1 per month for about 250GB of data (prior to compression / encryption).
for those unfamiliar to restic, you can mount a r/o snapshot via fuse for ease of access as well as the command line tool to programmatically access the repo.
Why don't you let restic upload directly to backblaze? I am sure you know this is possible, just wondering on why you made that choice, not trying to be a smartass ;).
I did at first but the latency was a killer; it took about 2~3 hours to make a snapshot. If I did the snapshot locally and then used rclone to back up the restic repo to BB then the whole operation took about 15~20 minutes, and the snapshot itself took less than 5; if the rclone gets interrupted you can just restart it.
I'd be wary if a backup programm doesn't strongly distinguish between the logic (the command-line client) and the gui. I cannot tell if this one does, but since it tries to "mimic [Time Machine] as closely as possible", I guess it does not.
There's a dbus layer which defines the actual backup operations, so without actually diving into the code to confirm this - I suspect there's a reasonable separation.
As does ZFS, unless I'm missing some distinction of "first class" in ZFS.
I'm really looking forward to Ubuntu 20.04 ZFS support. If I can use crypto with it (either the ZFS crypto or LUKS stuff), I'm going to switch my workstations over to it. ZFS snapshots work so, so well.
One thing I've really wanted from HAMMER is the deduplication. I've always had problems with ZFS dedup. I tend to really want it on my backup servers, but the DDT requires pretty amazing amounts of RAM. HAMMER's dedup seems less RAM-hungry, but I've only run small tests of it.
I've been using rsnapshot on /home for about 8 years. I run it once an hour in a cronjob. After using NetApp file servers with .snapshot dirs it is great to have something that kind of emulates it.
One suggestion for the author of that webpage, and a common complaint I have with open source software websites in general: tell us what your software does. I haven't used a mac, I don't know what features time machine has in any sort of detail so telling me it's a clone is pretty useless. Give a one-or-two sentence saying what it does, and then some bullet points of key features.
There's more description of the name than there is of the functionality.