Combination of both. Airsoft is a game, and toys are used in games. They're typically mass produced with cheap materials, and indeed a customs appeal I have categorized my MP5 trigger group as a part for a toy gun. Therefore it's not a stretch to visualize airsoft guns as toys, especially when compared with their real steel equivalent.
Then there's the demographic - you don't see a lot of people over 30 playing airsoft. There are many but in general it appeals to younger males, who in general don't have lots of money. If they can save a buck they usually will, not realizing that the bitterness of low quality will linger long after the sweetness of low price has faded.
That's my guess.