People usually don't carry pure gold and silver coins in their wallets.
If I put some in my wallet, would it be likely that airport security would detect some anomaly with my wallet as compared to other wallets?
Can physics answer this?
The physics answer would be each different metal coin can be distinguished through Xray spectroscopy, because the energy levels of inner shell electrons are unique to each element. See this reference for examples of Xray spectra: http://www.amptek.com/xrapps.html#coin
Whether or not a particular airport has such technology and whether or not they are interested in using it on your wallet isn't really a physics question.
The X ray detectors work by sending a beam of X rays through the stuff they are probing. The higher the density, the more X rays gets absorbed, the less one detects on the other side. In fact, we are not measuring density, but integrating it along the line of sight; the total mass the ray had to go through.
The main difference between gold and coins (mixtures like Ni-Cu-Al-Fe) from the X ray point of view is that gold is more dense, so more opaque to radiation for a fixed thickness. A single X ray cannot tell you that, but as they usually have two beams, we can get a quite accurate 3D reconstruction.
So, let's compare a gold coin with a nordic gold (what yellow euro coins are made of). The density of gold is $19.3 g/cm^3$, and nordic gold $1.7 g/cm^3$ (I couldn't find it in a table, but I computed it from dimensions and mass). If our coin has a thickness of 3 mm (I would say it is thin for the soft gold metal), we have the following cross-section densities:
Quite a difference. Would that show up on the screen? I have seen the cable to charge my phone. Assuming it is made out of copper, and a thickness of $1 mm$, we get an upper bound for our cross-section detection limit of (roughly) $0.8 g/cm^3$. So, yes, we can tell them apart.
Let's make it more difficult. Assume the coin is copper. It's density is $8.96 g/cm^3$, still a factor of 2. It's cross section density would be around $3 g/cm^2$, well above our threshold.