Hymns (and I assume you're talking about the old, standard hymnals many churches use) are often quite a bit more complex than your standard I - IV - V chords. Another thing that makes it a challenge is that you can often have a chord change for each note of the melody. But because they are usually written in only four voices, they are a great way to practice sight reading, especially for chords. Ages ago (insert your favorite "here goes David about his age" joke here), when I was first learning guitar, I'd play along at church by reading the chords using only the notation. It initially takes time, but you soon get the hang of recognizing chords in this format.
It certainly helps if you get to know which hymns you're playing ahead of time. You can then figure out a lot of the chords and puzzle out the ones that might be tough (sevenths, sixths, augmented and diminished chords).
Another thing, a lot of hymns in the old standard hymnals are written in distinctly non-friendly guitar keys (Bb, Eb, Ab and even Db), so you might also want to think about brushing up on your transposing and capo use.
There are any number of articles here at Guitar Noise that can give you the basics for help with this. Just write if you want some more direction.
In the following web page, I've found the sections "composing a melody to a given harmonic progression" and "harmonizing a melody" very helpful when it comes to playing along to hymns:
Our hymn book has only the vocal melody line, usually not the piano music, so it's a struggle to play along without chord notation. I try to harmonize, but I can't do it well on the fly. With some woodshedding I can figure out a bass line or guitar chord progression that works, and I'll write it down (on a copy, not in the book! :lol: )
"Everybody got to elevate from the norm."