Back in the mid-80s, when I was just a little kid, I remember there was this game show on TV where actors from TV shows would race laps against each other in an Olympic sized swimming pool. I remember one episode had Alan Thicke on it when he was on "Growing Pains".

Anyone remember the show and know what it was called?

I haven't thought about that in years.  I don't know if what you watched was the same as what I remember.  I lived in New York from 79 - 82 and they had this show on called Battle of the Network stars (usually once a year), and it was some crazy thing where the stars of TV shows formed teams based on what network they were on, for example it was NBC versus CBS versus ABC (I was always rooting for NBC).  It was truly great television and there seemed to be a lot to comraderie between the actors.  In the same vein I also remember one morning in the late 80's watching Charlie and Martin Sheen take on Michael Jordan in this Basketball skills thing.

Yes it sounds exactly like Battle of the Network Stars. I used to watch it at weekend lunchtimes. For some weird reason I distinctly remember Lorenzo Lamas being in one of them.  :D