I don't know if 'church' hymn books has the letters listed above the chords, but a
book of hymns may. You just have to look for it.
One titile, I believe, is "Then sings my soul".
In addition, if you know your chords, you could just write them in the hymnal, AS LONG AS IT'S YOUR HYMNAL!