If my husband has had public defender for 3 months but only talked to him twice in that time, how can he get him replaced?

You can ask for substitute counsel but getting a new public defender appointed is not automatic. You'll need to request a hearing. That means that you can get a different public defender to represent you so long as the judge presiding over your case can be convinced that this is necessary in order for you to put forth a competent defense. 
To obtain a hearing, you will have write a letter to the judge requesting a meeting with them regarding your appointed attorney. A copy should be sent to both the lawyer that you want removed and the prosecutor and make sure both copies and the original (which goes to the judge) are sent certified mail, return receipt requested; you also need to keep a copy for yourself.  Make sure that in it you state why you want present counsel removed. Be sure to need list all problems that you are having with your attorney but keep the letter short and to the point. Be as truthful as you can since you don't want to jeopardize your position before the judge by making false claims.
Further, you can contact legal aid and see if they would take the case. If not, check to see if there is a law school nearby to where you live; they typically run free/low cost clinics that handle these type cases. Also, contact the local Bar Association in your county; they may have a list of attorneys who will take his case "pro bono" (for free) or at least for a reduced fee income.