I copied a few definitions that I found online. This first one is from an online legal dictionary. It is a description of decrminalization:

decriminalization
n. the repeal or amendment of statutes which made certain acts criminal, so that those acts no longer are crimes subject to prosecution. Many states have decriminalized certain sexual practices between consenting adults, "loitering" (hanging out without any criminal activity), or outmoded racist laws against miscegenation (marriage or cohabitation between people of different races). Currently, there is a considerable movement toward decriminalization of the use of some narcotics (particularly marijuana) by adults, on various grounds, including individual rights and contention that decriminalization would take the profit out of the drug trade by making drugs available through clinics and other legal sources.

This next one I found in a google search for decriminalization verses legalization. It uses prostitution as an example:

Legalization: to make legal; authorize

Those that legalize prostitution (and yes, there are some places that do) “do so by giving the state control over the lives and businesses of those who work as prostitutes. Legalization often includes special taxes for prostitutes, restricting prostitutes to working in brothels or in certain zones, licenses, registration of prostitutes and government records of individual prostitutes, and health checks, which often means punitive quarantine. The term legalization does not necessarily have to refer to the above sorts of regulations. In fact, in one commonly accepted definition of legalization, legal can simply mean that prostitution is not against the law.” [quoted from this website] And this is not what we want. Prostitution should still be against the law – it just should not be a criminal act.

Decriminalize: to eliminate criminal penalties for or remove legal restrictions against (typically referring to consenting adult sexual activity)

While there are many reasons one gets into prostitution and many reasons one stays, I don’t think that putting these men and women in jail is ever really the answer. The times I have spoken with my girls after they have been in jail for a couple of weeks for getting picked up for prostitution I see no real effect the jailing had on them (other than pissing them off and making them annoyed). And to be real frank, I think it is the John’s that need to be put in jail if we are going to send anyone there. The laws that are currently in place for prostitution are reaching the “branch” problems – and we need to get to the root. Our laws are affecting the women involved while the men who are customers reap no consequences.

The criminalization of prostitution also makes it very difficult for these women to seek help or report violent actions against themselves to the police. If they report anything they can be charged or put in jail because of the actions that caused the crime against them. Several women I work with would be fearful of loosing custody of their child or losing status with the family and friends.

As you can see, there is a major difference in the two things. To decriminalize something would still mean it would be illegal but it would take away it's criminal status. To legalize it would make it perfectly ok to do.