I'm by no means an expert, but my suggestions would be:
Sign up in
http://www.voice-overs.com/forum/ ... <disclaimer, I am not signed up there> they do seem pretty friendly, and I bet if you rummage around the forums you'll find a lot of people asking the same question and a range of answers which might be useful.
Check out some of the podcasts from
http://podcasts.voices.com/voiceoverexperts/ (scroll back through the archives, and there are a number of shows dealing with setting up in business, pricing, marketing and so on. Scott Brick's recent one on audiobook demos was great.) Voices.com is another place to sign up as a jobseeker.
Record lots! I've done a few bits of non-LV work so far, which have come from people hearing my voice at LibriVox, and asking if I could just do something for them. I don't think it's the ONLY way to build a career, I certainly think you'd need to the legwork in terms of marketing and networking, but it's nice to have a ready portfolio to point people at, and as you say, it keeps your skills sharp. And it's nice to have a supportive community around, too, audio work can be kinda lonely.
Is there anybody in libri-land I could ask questions of about voice-overs? I have no prior acting experience, but took v/o lessons in a professional studio around 15 years ago. I have a really mellow tone (was told it was VALIUM FOR THE EARDRUMS) 
. I would like to get an in-the-trenches point of view about it, before I decide to run off and join the circus 
instead! My testfile is attached to give you an idea of my voice.