Indeed, a headphone amp is an excellent buffer also if the gain can be set sufficiently. QRV05 can be used as a very small solution but you will get some offset voltage because these buffers aren't any precision devices but the value of the offset is hardly harmful. It's a couple of mV.
If you of some reason prefer an opamp+buffer solution the QRV04 is excellent. Mind though that BUF634 is really hard to get but LME49600 is a second source, should work but I haven't tested it yet. QRV07 is also excellent and TPA6120 is easy to get and I have it also for sale.
If you'll choose QRV04/07 you don't have to worry about the PS since they are integrated on the pcb, otherwise JSR06 would be a good choice.
The transistors in the QRV05 don't have to be matched but it doesn't harm either.
The QRV05 is an advanced emitter follower therefore the gain is always slightly under 1. No to you question.