You need to get rid of them, they can do a lot of damage. The ones you see are easy to kill, just pick them off.
The problem is, if you have a few, you probably have many, hiding on your other pines. Bring out the big guns, go on a sawfly larva hunt.
When selecting a pesticide to use, be certain the label lists sawfly. If the label does not specifically list sawfly, then there is no guarantee it will work.
As above, BT works on caterpillars (lepidopterans) but not on fly larva (Dipterans).
Some organic options, there are Spinosaids that work of Dipterans in organic formulations for household use. On the farm we use Entrust, labelled organic, but that label might not be approved for household use even though it is organic.
CHeck out the avermectins, another group of organic pesticides that may have some labelled for household use.
WHat is approved varies from state to state and country to country, so I do not list specific brand names. You will have to do your homework to get your local list of approved chemicals.