Despite their showy yellow flowers, goldenrods aren't the culprit for seasonal allergies. The inconspicuous ragweed plants reaching 10 feet tall spread pollen widely by wind.