The simple answer to "why are asymmetric algorithms more costly than symmetric algorithms" is because we have more requirements for asymmetric algorithms, and those additional requirements cost something to satisfy.
The requirement for (say) a symmetric encryption algorithm is "if you encrypt a message with the key, then someone without the key cannot infer anything about the message (apart from the length), but someone with the key can recover the exact message".
Now, asymmetric algorithms also meet this requirement. If we replace "key" with "the public and private keys", then we can encrypt messages (with the public key portion), and no one without the private key portion can deduce anything about the encrypted message.
On the other hand, we place additional requirements on asymmetric algorithms; someone with the public key still cannot decrypt messages, even though they can encrypt. This implies that the public key and the private key must be related (because they are, in some sense, inverses of each other), but it can't be in an obvious way.
This means that the efficient constructions we use for symmetric algorithms (xor'ing in a keystream, Feistel networks, or SP-networks) just won't work, because those are easily invertable (given the key) by construction. For symmetric operations, we can pick operations rather arbitrary, limited only by their efficiency, and how well they are disguised by a secret key. Instead, we also need to limit ourselves to operations which have nonobvious relationships; these mathematical constructions (which is where we find this nonobviousness) are considerably less efficient.
As for when asymmetric operations "trump" asymmetric ones; the rule is simple: if you need the additional properties that asymmetric operations provide, they're better; if you don't, then they're not needed.