So, you want to build a banshee.
The pistol shrimp is an interesting example, but ultimately won't work. It snaps open its claw so quickly that a cavitation bubble is created. This creates an acoustic wave of 80kPa (218 dB) at a distance of 4 cm from the claw. It's enough to kill small fish.
However, underwater acoustics are less impressive in air, so this won't directly translate.
In the air, against higher-order vertebrates, there are only artificial sonic weapons. The best examples have been used for deterring pirates and dispersing crowds, they are called long range acoustic devices.
To make this most effective for an animal, it would be best to use ultrasound frequencies.
Studies have found that exposure to high intensity ultrasound at frequencies from 700 kHz to 3.6 MHz can cause lung and intestinal damage in mice. Heart rate patterns following vibroacoustic stimulation has resulted in serious negative consequences such as atrial flutter and bradycardia.
That's frequency, the other component would be the sound pressure.
Tests performed on mice show the threshold for both lung and liver damage occurs at about 184 dB. Damage increases rapidly as intensity is increased.
Now, dB is not a unit of pressure, it's a ratio of pressure, and thus unitless. But we can convert it to pressure easily. Additionally, saying 184 dB doesn't mean a lot without an associated distance, but we're stuck without that information. All we can calculate is that about 31.6kPa is being exerted somewhere along the travel from the transmitter to the receiving animal (most likely, though, it's at the surface of the animal).
From this we can assume your creature will need to generate less than 184 dB acoustic waves in the ultrasonic range and be close range to the target for an attack.
Note about volume
It's important to point out that 184 dB is really loud. The loudest non-bass acoustic instrument appears to be the trombone, peaking around 114 dB. Many times quieter than required. Some bats have been recorded at 135 dB. The loudest land animal is possibly the howler monkey, topping out around 140 dB. I know it seems like 140 dB is pretty close to 184 dB. It's not. Decibels are a logarithmic scale. Every 3dB increase is a doubling in power. In air, 140 dB is only about 200 pascals (rms) while 184 dB in air is 31,698 pascals (rms). A 140 dB sound is like a jet engine 100 feet away, 185 dB kills hearing tissue, and 194 dB is the loudest sound possible in our atmosphere.
It might not be impossible that a creature could produce such a sound. But it's highly unlikely. It's orders of magnitudes higher than any existing biological sound. However, that is the maximum before permanent damage occurs. For your creature, it needs significantly less than that for stunning alone.
So, such a creature is entirely in the realm of possibility. Even a howler monkey could disorient you with a scream if it had a mind to.
With that background, on to your questions.
Can this be accomplished biologically?
Yes, a set of lungs and an enlarged hyoid bone will do the trick.
What would the mechanism for such a skill be?
It would likely be vocalization. Though other possibilities exist, stridulation comes to mind, and that would certainly look menacing.
How would this animal protect itself from its own abilities?
It's a good bet that the creature itself is deaf or in the process of making the sound, temporarily deafens itself. Bats do this. When bats call for echolocation, a tiny muscle holds their bones of hearing in place while they echolocate so they don't deafen themselves.
Are there any physical limitations having this power would necessitate?
This depends on the method used to create the sound. The most effective would be an array of sources, either vocal or stridulatory organs. This would allow for more directionality and focus. This would make the attack more effective over a longer distance and use less energy to generate.
It's really easy to avoid this attack. Simply wear ear plugs. Better yet, use active ear plugs, they simply don't transmit any sounds above a certain decibel level. Deaf people are inherently immune.