James Vassar sold his farm and settled in the center of Poughkeepsie to brew ale. He built a brewery, in a part of which the family lived while a house was built for them nearby.
Matthew Vassar, with his mother's aid, ran away from home to avoid being apprenticed to a tanner. "Set off to seek my fortune with 6/ in my pockett, two corse East India Muslin Shirts, a pair of woolen Socks, Scow Skin Shoes, all tied up in a cotton bandana Handkerchief."
He found employment at Butterworth's store at Balm Town, near Newburgh. “…they pointed out my work if I did Stay, viz, to Measure Wood Weight Iron, measure Salt, &c, in fact do all kinds of drudging even to taking care of the Old Gentleman’s horse. All of which I consented and found a home for 3Years, Begining with Labour for a living and ending on a Sallary of $300 pr annum.” Elizabeth Hazelton Haight, ed., Autobiography and Letters of Matthew Vassar