There are two schools of thought here. There are those who say that telling the truth is all that counts and that if you're honest with him, he'll forgive you. Then there are those who say that what you don't know doesn't harm you.
We're not endorsing lies here, but if you want your relationship to survive it might be better not to tell him. This could well apply if your affair is well and truly dead, buried and over with, and you don't want to break up your home. You can admit it to yourself but you might not want to put him through the truth.
Some people can't live with the guilt and prefer to clear their conscience by telling all. In which case, you might have a clear conscience but he might be completely destroyed by the truth. There are very few relationships that can bounce back from a confession of infidelity - they say the damage has been done.