Ok, so in a bid to resolve this, I started from scratch. I've made sure I've enabled debug logging and I dumped the database and started from scratch.
http://pastebin.com/tekWieJJ is the debug log - as you can see it never even looks at movie.nfo.
I also ticked 'movies in separate folders named like the movie'
EDIT - OK, so this turns out to have been somewhat of a wild goose chase - I realise now that the issue is down to my setup. I don't want to touch the original files/folders so I've created a duplicate structure which is symlinked back to the original files. In theory this should work fine, however XBMC doesn't look in the directory it's given, it traverses the directory tree back to the original file location and looks there instead, hence why it wasn't finding my movie.nfo. I've since verified this by hardlinking the 21 Jump Street folder, coping the nfo in there and scanning that - it functions exactly as expected.
Many thanks to jmarshall for his input in helping to resolve a problem which is ultimately of my own making.