Vmtouch – Portable file system cache diagnostics and control

[old-gregg]

https://github.com/hoytech/vmtouch

[TheAceOfHearts]

I think linking to the homepage would've been better. It includes a "What is it good for?" section, as well as examples.

The articles on the vmprobe[1] homepage are also very well written. They present a simple scenario, and talk through it without lots of buzzwords. A bit sad that there's no macOS support.

I just tried using vmtouch on the root of a web project with lots of dependencies. I tried evicting and touching a few times, and it seems to consistently take ~1 second off its startup time. Unfortunately, touching seems to take ~3 seconds.

Does anyone regularly use this in their daily workflow?

[0] https://hoytech.com/vmtouch/

[1] https://vmprobe.com/

[hoytech → TheAceOfHearts]

Thanks! Ya I'd like to add macOS support at some point to vmprobe, but I'm using a bunch of linux-specific stuff at the moment: mostly pagemap/kpageflags, although also some mmap/madvise flags that are linux extensions. Still, the basic use cases don't need this.

Regarding your web-app, it's possible that you are touching a bunch of files that you don't need to. Rarely-accessed static files, the .git directory, and so on. vmprobe snapshots might help here, or you could figure out which files are used for start-up and just touch those. There's also the -I switch. You might try something like:

  vmtouch -t -I '*.html' -I '*.js' -I '*.css' /var/htdocs/

Also, sometimes it is possible to page in the necessary files before your web-app is started, or concurrently while it's starting (maybe at boot-time, container start, etc). Also, sometimes assets are paged in before the first request is received.

I know of quite a few people and companies that use vmtouch in their daily workflows. vmprobe is still largely experimental at this point though.

[microcolonel → TheAceOfHearts]

For what it's worth, the homepage is basically the same document as the README.md.

[martythemaniak]

vmtouch and vmprobe are both pretty cool utilities. There's an interesting presentation where you can see how it's used: https://www.youtube.com/watch?v=yoBIr9sFCS8

[currriuosly]

Where can I read more about Linux internals? Any good resource with design decisions and algorithms?

[hoytech → currriuosly]

In terms of virtual memory, there's a somewhat dated book that explains the linux implementation:

https://www.kernel.org/doc/gorman/pdf/understand.pdf

And there are some maintained text files in the kernel source tree itself:

https://www.kernel.org/doc/Documentation/vm/

Of particular interest is the pagemap.txt file.

There's also a really neat utility in the kernel source called page-types and you can learn a lot by compiling it and playing around, and of course reading the source (there are lots of tricks in there):

http://lxr.free-electrons.com/source/tools/vm/page-types.c

Finally, you could also read the source to vmtouch or my vmprobe utilities, or ask questions in the vmtouch github repo -- we can probably point you in the right direction.

[alpb → currriuosly]

https://www.gitbook.com/book/0xax/linux-insides

[thedjinn → currriuosly]

I read Understanding The Linux Kernel 3rd edition a couple of years ago and that book covers quite a lot of it.

[mark212]

Now _this_ is how you write a readme. The first sentence has a clear, concise explanation of what the code is for and why you might care, without a garble of acronyms and abbreviations.

[ausjke]

Great tools that are very useful for my purpose, installing it now and Thanks!

[boomskats]

I'm so happy to have come across this. Had no idea it existed.

[hoytech]

Hi! I've also been working on another program called vmprobe which does some neat things vmtouch doesn't:

https://vmprobe.com/