Individual Details

This man could not have been born in Virginia if he was a son of Thomas Spencer and Esther Andrews.... His birth also seems a few years too early to have been a child of this couple.

And, in fact in one database, the immigrant to Virginia was Abraham Spencer, granted land in Martin's Hundred, 1638, died about 1655. His son, born by say 1737, married Anne Woodward. Their possible children:
Susannah, b. 3 Nov 1686, d. 4 Nov 1686, St. Peter's Parish
Thomas, b. 1689, died by 1734, but had a son Woodward Spencer who required a guardian, Hanover Co, 1734.
Possibly - although continued research shows even the connection to each other is doubtful and they were not likely either a son of Thomas and Anne.
Benjamin Spencer who was present in St. Peter's Parish, died Louisa Co, whose wife was Ann and they had children John, Thomas, & Elizabeth. Their son Thomas went to Caswell, NC where he died in 1781.
Abraham Spencer who married who married Susannah Sharp

Birth	1659	King William County, Virginia
Marriage	Ca 1680	King William County, Virginia - Anne Woodward
Death	20 Jul 1708

Spouse	Anne Woodward (1662 - )
Child	Abraham Spencer (1700 - 1762)

Father	Thomas Spencer (1635 - 1689)
Mother	Esther Andrews (1641 - 1698)