Hacker News new | past | comments | ask | show | jobs | submit login
Libtree: Turns ldd into a tree; explains why shared libraries are found or not (github.com/haampie)
289 points by fanf2 on Dec 3, 2021 | hide | past | favorite | 57 comments



Got me to learn that vim requires libcanberra requires libvorbisfile ... libogg

    ./libtree_x86_64 $(which vim)
    vim
    ├── libtinfo.so.6 [ld.so.conf]
    ├── libselinux.so.1 [ld.so.conf]
    │   └── libpcre2-8.so.0 [ld.so.conf]
    ├── libcanberra.so.0 [ld.so.conf]
    │   ├── libvorbisfile.so.3 [ld.so.conf]
    │   │   ├── libvorbis.so.0 [ld.so.conf]
    │   │   │   └── libogg.so.0 [ld.so.conf]
    │   │   └── libogg.so.0 (collapsed) [ld.so.conf]
    │   ├── libtdb.so.1 [ld.so.conf]
    │   └── libltdl.so.7 [ld.so.conf]
    ├── libacl.so.1 [ld.so.conf]
    ├── libgpm.so.2 [ld.so.conf]
    └── libpython3.8.so.1.0 [ld.so.conf]
surprising

(and super neat tool, even though people mentionned ldd could do part of it)


https://github.com/gentoo/pax-utils/blob/master/lddtree.py already does this. Cool to see though, very useful


Seems like it uses ELF parsing, I wonder if it works on non-native binaries (lddtree does not)


And https://github.com/ncopa/lddtree is a single-file bash re-implementation.


lddtree is much faster, but libtree has more detailed info.


If you want something fast, note that (https://github.com/haampie/libtree-in-c) is currently faster than ldd:

    libtree-in-c $ make CC=musl-gcc CFLAGS="-Os -s -static"
    musl-gcc -Os -s -static -c libtree.c
    musl-gcc -Os -s -static -o libtree libtree.o

    libtree-in-c $ hyperfine 'ldd libLLVM.so' './libtree ./libLLVM.so'
    Benchmark #1: ldd libLLVM.so
    Time (mean ± σ):       6.1 ms ±   0.3 ms    [User: 5.2 ms, System: 1.2 ms]
    Range (min … max):     5.3 ms …   8.1 ms    348 runs

    Benchmark #2: ./libtree ./libLLVM.so
    Time (mean ± σ):       2.4 ms ±   0.4 ms    [User: 1.2 ms, System: 1.2 ms]
    Range (min … max):     1.7 ms …   3.4 ms    840 runs

    Warning: Command took less than 5 ms to complete. Results might be inaccurate.
    
    Summary
    './libtree ./libLLVM.so' ran
        2.54 ± 0.43 times faster than 'ldd libLLVM.so'


The C rewrite was released in libtree 3.0.0


Or even more basic:

  $ eu-readelf -d /bin/ls | grep NEEDED
  NEEDED            Shared library: [libselinux.so.1]
  NEEDED            Shared library: [libcap.so.2]
  NEEDED            Shared library: [libc.so.6]
Of course the trick is doing it recursively and having nice output.


That's not the same. ldd actually finds the libraries and their dependencies.


Of course. I'm just pointing out how the source data is encoded in the ELF file. As I said in the comment there is much value in the tools that take this data and the locations of the libraries and build up a nice tree.


It's not though, an ELF file only lists its immediate dependencies. To find the complete tree you actually need to find the dependencies and parse those, which is exactly what this does.


On Solaris, you can get the tree with:

    ldd -s <binary_name>
It gives you the dependency tree, tells you the search paths it used for each dependency in the tree (LD_LIBRARY_PATH, the RUNPATH for the specific binary, and ld.config file), and the reason it rejected any files it did find, e.g. missing symbol version.


See also lddtree from pax-utils: https://cgit.gentoo.org/proj/pax-utils.git/tree/lddtree.py


Ah, excellent. Windows has Dependency Walker, which was always useful for tracking down weird DLL errors - glad to see something like this for Linux too.


You'll probably want to use this: https://github.com/lucasg/Dependencies


Beware that ldd needs to run the binary to check for dependencies. It can be a security risk to use it without a sandbox.


Huh, any reference about this? How much of the binary does it run?


Now I'm wondering why ldd doesn't offer this feature in the first place.


The output format isn't nice, but LD_DEBUG=files will tell you which file requested each library.

(The LD_DEBUG environment option is a useful tool in the rare circumstances when you really, really need help debugging the library loading process.)


Because it's a core Linux tool and they basically never change. Why doesn't ps have an option to only show your processes? Why doesn't objdump tell you what all the symbol flags mean so I don't have to look them up on StackOverflow every time?

I'm sure there are better examples but there are a ton of ways the usability of the Linux CLI could be improved but nobody is going to bother because everything's written in horrible C and to contribute you'd have to join some mailing list and convince a load of stuck-in-the-mud it-was-hard-for-me-so-I'm-not-making-it-easy-for-you neckbeards that your change won't break compatibility on a PDP-11, and anyway what's the problem? You just run `ps -fu $(whoami) | less`. Easy.

Way easier and more fun to write a new tool.


> Why doesn't ps have an option to only show your processes?

Because that is the default behavior.


No, by default it shows you processes for the current terminal.


Only your processes for this terminal.


Yes. I meant "show me all my processes". A very common task that doesn't have a good solution.


Sure it does: “ps x” does that.


I agree that the interface of ps is awful with its two conflicting ways to define options, but `ps u` does show all your processes on Linux. I usually type `ps fux` to get a nice tree. Sadly that's Linux-specific and doesn't work on the Mac.


`ps u` just changes the output format. It's not the same as `ps -u $(whoami)`


You are right! I misremembered. It's `ps x`.

From `man ps`:

       x      Lift the BSD-style "must have a tty" restriction, which is imposed upon the set of all processes when some BSD-style (without "-") options are used or when the
              ps personality setting is BSD-like.  The set of processes selected in this manner is in addition to the set of processes selected by other means.  An alternate
              description is that this option causes ps to list all processes owned by you (same EUID as ps), or to list all processes when used together with the a option.


Trees are quite hard to do well in a CLI app. Everything from the Unicode drawing to the sizing to the color choices. This one looks great.


It's not like ldd is a paragon of cli design to begin with. Indentation isn't difficult, you just pass in the indentation level and increment as needed as you recursively walk the tree.


Unicode isn't necessary if the terminal supports ACS.


You can do a passable job with just + - and | if you need to. Have done that for a ucontroller's cli.


What is ACS?


Most terminals (even non-unicode-capable ones) offer a feature to switch into "alt charset mode" which provides glyphs for drawing lines.


Thanks! It’s getting rare to learn about a feature that surprises me, but I had no idea about this.


just saying: pstree figured it out


Is there anything remotely close to this for macOS?


You can use things like `otool` which will tell you what path to the library the binary is expecting. Here's a tool I wrote to fix paths recursively for an application, for example:

https://github.com/jveitchmichaelis/deeplabel/blob/master/fi...


Check out https://github.com/kwin/macdependency


I like your other implementation more, libtree-in-c. Actually, I think it is better to exchange the names and let's libtree be in C and the other one be called libtree-in-cpp or libtree-in-c++.


Yes, I agree, this is the plan :)


Nice! reminds me of copydeps (https://github.com/Genymobile/copydeps)


On Linux with glibc, ldd -v will give you similar information, though less pretty of course.


SUPER GREAT for bundling software into a folder.

Great for demo and standalone prototyping.


This is very cool!


What? new CLI utilities are not in rust?

And yes it's very useful, can do more than pax-util's lddtree(libtree can help packaging your binary with all its libraries in one package), just installed and looks great.


Please don't do this. There is nothing that says CLI utilities (or any other programs for that matter) HAVE to be written in Rust.

This type of comment puts me off Rust, slowly but surely.


It is a huge assumption to think the parent comment was from anyone in the Rust community. It could be a Rust hater.


I took the comment as a jest.


It is just a harmless joke, sorry if it offends anyone.

I use a lot of rust CLI tools myself and love them. Meanwhile I also do c++ coding. they complement each other well.


If some bad apples in a community (every community has them) can make you give up on a technology then you never actually considered it in the first place.


I doubt that the grandparent is a Rust developer.


Where are you seeing rust? Or is this just tongue in cheek?


sorry, typo, forgot 'not' there


This is fine. It's a great beginner project -- C++ is probably painful even statically compiled, and Go/Rust would be better.

But you may as well make it a little smarter. Over a decade ago, this utility could be done ad-hoc in 100 lines of Perl (including spacing and comments), and that includes "try to see if yum on RHEL6 knows where to find the missing dep, build a list, and prompt to install": https://gist.github.com/evol262/3d67c4295bbe78135a4927bd1d82...

It would be a fun project for you to try to do this with dpkg/pacman/dnf (which is basically the same as yum for syntax, just that you're not gonna be a sysadmin hacking out Perl to solve a problem)


> C++ is probably painful even statically compiled, and Go/Rust would be better

What does that even mean?


That code does not work, it won't even compile with `-Mstrict`.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: