Not sassafras, too many leaf lobes.
If tree like, with examples over 10 feet, most likely Crataegus, hawthorn, there's many species native to north america, and just as many natural hybrids. Finding a Crataegus that "doesn't quite fit" a species description is more the normal case, than not.
If the "tree" seems to be shrubby, remaining a bush more than a tree, check out genus Ribes, the currents and gooseberries. Again, many North American species.
Both have distinctive fruit and flowers, though for Ribes fruit are eaten by wildlife usually by the time it is ripe.