Using higher primer concentrations means more primers in the mix, which increases the chances of primers binding to partially complementary sites, resulting in nonspecific PCR products.
On the other hand, using too low primer concentrations can result in low or noamplification of the desired target.