At the time of Olive's death, Jesse was faced with raising 4 boys ranging in ages from Chester, who was 13 to Orrin, who was barely 4 years old. Jesse chose to marry Ruth Brooks on 6 Oct 1803 in Middlebury, Addison, Vermont, a nearby town, to help take care of the house and help raise the four boys. They lived together in Waltham until they both died there in 1839.