Sounds like you are getting the hang of it!
Yes, the arduino ground connects to the ground of what you are measuring. If that is a single supply op amp than that is what you connect to.
For example, an op amp like this, which runs from +5v and Gnd.
The arduino running on batteries is like a portable multimeter - you are measuring the red lead (Analog in) with respect to the black lead (arduino ground).
Only the arduino can't withstand negative voltage, while the multimeter can - so the black lead must be connected to the circuits ground and only positive measurements can be made.
If you have an op amp with +/- supplies, then you must be careful - perhaps adding a diode before the analog input pin to rectify the incoming voltage, at the loss of voltage drop across the diode.