I haven't had good experience with modern post-publication peer review so far.
My biggest complaint is that few know those comments exist. For example, I commented on one paper to highlight methodological problems that make it impossible to trust their results. (They developed a new algorithm for X. They compared it to the naive implementation. The naive implementation was poorly coded. Most of the speedup disappears by in comparison to a well-written implementation.)
I've talked to a few people about this exact paper. None noticed the link to the comment page.
I say "modern" because post-publication peer review isn't new. One of the older mechanisms was the letter to the editor. Those letters (at least in the ACS journal I'm thinking of) have a DOI and are searchable. A few of these letters have proved useful to my research.
But the modern post-publication peer reviews don't have a DOI and aren't indexed by Google Scholar or other systems, so they are less useful than a old Letter to the Editor.
(I once asked about sending a Letter to the Editor to an Open Access journal, concerning problems in one of their publications. They said they don't support those sorts of short communications, and my only option was the $1,500 to publish a full paper, which would have to go through the normal peer review process.)
My biggest complaint is that few know those comments exist. For example, I commented on one paper to highlight methodological problems that make it impossible to trust their results. (They developed a new algorithm for X. They compared it to the naive implementation. The naive implementation was poorly coded. Most of the speedup disappears by in comparison to a well-written implementation.)
I've talked to a few people about this exact paper. None noticed the link to the comment page.
I say "modern" because post-publication peer review isn't new. One of the older mechanisms was the letter to the editor. Those letters (at least in the ACS journal I'm thinking of) have a DOI and are searchable. A few of these letters have proved useful to my research.
But the modern post-publication peer reviews don't have a DOI and aren't indexed by Google Scholar or other systems, so they are less useful than a old Letter to the Editor.
(I once asked about sending a Letter to the Editor to an Open Access journal, concerning problems in one of their publications. They said they don't support those sorts of short communications, and my only option was the $1,500 to publish a full paper, which would have to go through the normal peer review process.)