After working through my .emacs file I found that this problem goes away when I turn off cua-mode.
Not sure whether this is actually a bug or if it's the intended behavior.
Brian Tannahill wrote:
> (global-set-key (kbd "M-J") 'backward-word)
[...]
> Running describe-key (C-h k) shows that the bindings were
> successful. And these bindings move point as they should, but for some
> reason it sets the mark at my original position, and gives me a
> highlighted region as I move the point.
I'm unable to reproduce this; even with shift-select-mode on. Note that
this-command-keys-shift-translated says:
Shift-translation occurs when there is no binding for the key sequence
as entered [...]
Does this happen if you start from emacs -Q?