As a heavy Thunderbird user, my biggest complaint is offline support. An IMAP sync will download new emails' subject lines, but not the actual message bodies. Clicking on an email subject line then requires freezes Thunderbird for multiple seconds while it downloads the message body for that single email.
Right click the account and go to Synchronization and Storage. Double check on individual folders. I know this works because my ImapMail folder (in the profile folder) is much bigger than it would be if it only had subjects, and a quick sampling shows full messages.
If you have this on and it's still taking seconds, the problem is probably coming from something else.