Non-compete

Generally, non-competes are enforceable in your state; fairness is not an issue--the law doesn't care if contracts are fair if you agreed to them; and it's not a defense to a contract to say you were "enticed" to sign it (unless threats of force or illegal acts are used to coerce you, or there was actual fraud, or lies about important facts), since after all, everyone is "enticed" to sign agreements or contracts by being offered something to do so. 
However, while non-competes are enforceable, courts in your state do not like them and scrutinize them carefully if a lawsuit is brought to enforce them. The court could reduce the length or duration, the geographic scope, the types of jobs barred, and/or the penalty, if they feel such are excessive--beyond what is reasonably needed to protect the employer's legitimate interests in not having an employee immediately turn around and compete. In the situation you describe, where you had a very low level job which does not seem to have included having access to important confidential or proprietary information, or having been extensively trained by the employer, and the non-compete is very broad with a very high penalty for breach, there is a very good chance that if he were to sue you for the money under it, that a court would either invalidate the agreement or significantly reduce what he could get. However, that is not guaranteed--while courts do not like non-competes, as stated, they do enforce them, so if  you appeared before a judge with a *very* strict view of contracts (i.e. if you agreed to it, you are bound to it), it could be enforced. So you have a good chance if you break the agreement, but there certainly is a risk--at least that you'd have to pay something, even if not the full $40k.