There are a number of incompatibilities caused by outright different function names ("tfeestimate" in Matlab vs "tfe" in Octave for transfer function computations for example) or the order of input arguments in functions. Also, there are several functions in Matlab that don't exist yet in Octave.
Try downloading the AbraVibe collection of functions at http://bcs.wiley.com/he-bcs/Books?action=index&bcsId=632... and running them on Octave. Some of those functions require modification before they will run, especially functions generating graphical output.
Interesting. I find with small utilities that I download, Octave runs matlab fine. With bigger things (optimization packages in my case) you do hit incompatibilities. I guess I don't use the latter that much.