The active alkaloids in Psilocybe mushrooms (principally psilocybin, psilocin and baeocystin) are the main components responsible for the entheogenic effects. Of these, psilocybin is the most sought after and most stable psychedelic compound, although it is converted to psilocin in the body after ingestion.
Psilocybin and psilocin are both indolealkylamines and structurally similar to the neurotransmitter serotonin (5-hydroxytryptamine or 5-HT). Psilocin is a mixed receptor agonist, and mostly interacts with the 5-HT1A, 5-HT2A and 5-HT2C receptor subtypes, though unlike LSD not the dopamine receptors.
It is the 5-HT2-receptors in the body that are thought to be largely responsible for the psychedelic effects of consuming mushrooms, as they are particularly receptive to the hallucinogens found within them, although their role may not be greatly more significant than any other of the 5-HTs.
For Fly Agaric (amanita muscaria), the main active agents are muscimol (a psychoactive alkaloid not unlike psilocybin, but an agonist of the GABAA receptors) and the neurotoxin ibotenic acid; this can also produce entheogenic effects at higher doses, but the majority of it is excreted unaltered in urine.