The Dutchess County Colonization Society was formed for "colonizing the free people of colour, of the United States, with their consent and choice." Matthew Vassar was Recording Secretary.     Poughkeepsie Journal

The Marquis de Lafayette was entertained by the village of Poughkeepsie. En route to Staatsburgh to visit his Revolutionary War comrade-in-arms, General Morgan Lewis, Lafayette arrived by boat at 2:30 in the morning.  The village trustees, including Matthew Vassar, greeted him at the Forbus Hotel on Cannon Street, after which was entertained at breakfast at the Poughkeepsie Hotel by some 60 people—including the Founder—before continuing to Lewis's estate on the Hudson.  Edmund Platt, The Eagle's History of Poughkeepsie from the Earliest Settlements 1683 to 1905.

The story of Matthew Vassar's rise in the world began in tragedy, with of the destruction by fire in 1811 of his father's thriving brewery and the suffocation the next day from carbonic acid fumes of his father's partner, Matthew's elder brother, John Guy, as he was inspecting the ruins. Devastated by his losses, Vassar's father sank into a depressed state from which he never fully recovered, leaving the 19 year-old Matthew to support his mother and father and his brother's family and to rebuild the business. Aided by his sister Maria and her husband, George Booth, a manufacturer of woolen cloth, Vassar founded M. Vassar & Co., brewing ale in one of Booth's dye houses.

Recognizing the need to quickly reestablish the destroyed brewery, Vassar entered into a partnership with a prosperous Englishman named Thomas Purser, and a new building arose on the site.  In 1815, Purser sold his interest in the business to J.J. and N. Conklin, Jr., and on September 9, 1829, Vassar's partnership with the Conklins was dissolved. In 1832, Vassar brought Matthew Vassar Jr. and John Guy Vassar, sons of his late brother, into partnerships in the flourishing M. Vassar & Co.