These weren't innocent people who just made a simple, careless mistake. They had witnessed Moses perform many miracles, including the parting of the Red Sea, freeing them from Egyptian bondage, and guiding them to water and food in the desert. These Israelites willfully defied God - they openly and willfully rebelled against Him.
Furthermore, the Canaanite God, Moloch, is a god who is associated with child sacrifice. The golden calf illustrations look very, very similar to some of the depictions of Moloch, and of course the Isarelites were already heavily influenced by Canaanite culture (Moloch is first mentioned in Leviticus, so we're not too far off in the timeline). I don't know the full context, but I'd say it's very likely that the golden calf worshipers were likely to have followed some of the same Canaanite practices, namely sacrificing live, human sacrifices of small children. When you consider the fact that only 1 out of 200 people were slain (600,000+ left Egypt), it was actually a very small percentage. The Israelities were asked to affirm their beliefs and their willingness to stand on the Lord's side (Ex 32:26), so I can only surmise that the small percentage that were killed openly denied God, the teachings, and the miracles.