Being imperfect relative to the flagship commercial phones is a problem, because ultimately end-users have to use these things for them to remain going concerns, and it's hard to see which users are better served by a less secure phone stack.
That would be imperfection in terms of features, not security. And yes, users do need to be able to actually do what they want, but the bar isn't that high and it certainly isn't uniform; some of us are happy with web browser, SMS, and a shell. Further features will widen the appeal, but for starting out they seem to be doing fine.
I believe that it is at least an equally secure phone stack vs most Android (granted, with the root of trust in the user rather than manufacturer, which I think is a good thing), and only might lose against Apple and Google flagships thanks to hardware features.
"Versus most Android" is, of course, a dodge, because the Android ecosystem ranges from comically insecure phones to expensive devices that asymptotically approach the security of iPhones. Achieving parity with commodity Android devices doesn't help end-users; every user who uses a Librem device to obtain that weak level of security has probably been harmed by the project.
This isn't to say that it's impossible to build a device that is more secure than an iPhone! It isn't. Librem simply isn't doing it; they have other priorities, including feature parity with modern smartphones. They're not willing to make the serious tradeoffs needed to get security given their circumstances.
By "most Android", I mean "everything without a Titan chip". I'm not convinced that it's meaningfully less secure for anything but targeted attacks by abnormal skilled attackers. Should be better, if its kernel is better maintained.
Considering the plethora of ARM trustzone and bootloader attacks against Android devices I don't think an abnormal skilled attacker means what you think it means the average repair shop can extract data from the majority of android phones today.
You also have successful key recovery/bypass attacks against most non-hardware backed crypto Android devices as well.
Like Thomas said it's not impossible to build a secure device, it's not even impossible to build a secure open mobile device but it doesn't seem that they are doing it.
Their focus is on having feature parity, using commodity hardware and just having an FSF approved stack.
Having an FSF approved stack doesn't make you more secure by default.
And usability has a great impact on security, I remember the early android days where getting a file off the device was PITA so myself and many others were running an FTP server on the phone, and since most of our phones were rooted im pretty sure it was running as root.
The other side is things like permissions while mobile operating systems aren't that great still they've began taking application permissions really seriously going through the Librem documentation I don't see anything that is even remotely close to the level of granularity that Android and iOS offer today.
Sure they might add that in the future but the point being is still that there is little chance that the first phone they launch would be more secure than an android phone yet alone a modern iPhone in fact I would bet at least one paycheck that they would be considerably less secure at least initially, and then it's the question of how they would be able to maintain and support their platform given their size to begin with.
I don't doubt the intentions of the developers I just highly doubt that anything they set is even remotely achievable.