Captain Henry Brown inherited the land that would become Ivy Cliff plantation from his father. During the Revolutionary War, he served three terms in the Continental Army, suffering a leg wound at the Battle of Guilford Courthouse in 1781.[14] He also served under Colonel Charles Lynch of modern day Altavista, Virginia in his somewhat infamous “pursuit of Tories,” during which Lynch’s forces unmasked and tried a group of local Loyalists who planned to compromise Lynch’s saltpeter mines and strike a blow at the heart of the Southern Continental Army’s ammunition supply.[15] 

Captain Brown’s children remembered him as a loving and generous father who supported them selflessly in all of their endeavors. In 1829, his son John Thompson Brown wrote, “My dear father, I have but one poor recompense to offer you—the best and purest affections of a heart that loves and venerates you; and a fired determination so to conduct myself as to render, as far as it is in my power, the evening of your life honoured and happy. Long—long may you be spared to guide, advise and cheer us on our path through life and to receive our grateful and heartfelt benedictions.”[16] Captain Brown served as a local magistrate and sheriff in Bedford County and survived his wife and two children who lived to adulthood.[17] He died in 1841 at the age of 81 and is buried in the Brown family plot, which is now privately owned separately from Ivy Cliff’s remaining acreage. 

John Thompson Brown was undoubtedly one of Ivy Cliff’s most prominent residents. He was born (1802) and died (1836) in the house that still stands today. Following his untimely death at age 34, the Richmond Enquirer wrote, “Perhaps we hazard nothing in saying […] that the death of no man could have touched more truly the heart-strings of Virginians.”[18] John Thompson Brown served as a member of the Virginia House of Delegates, and he was even considered for candidacy in the U.S. Senate. At the peak of his political career, he actively debated states’ rights—notably when South Carolina threatened secession over interstate tariffs in the 1830s—and slavery. Given his upbringing on his father’s plantation, Brown’s opinions on this subject are particularly noteworthy.

Brown took an active role in the Virginia Slavery Debate of 1831-1832, during which the House of Delegates convened in the wake of Nat Turner’s Rebellion, the deadliest slave revolt in American history. At the heart of the debate lay the future of free blacks in Virginia, but one faction of the House called for the emancipation of the state’s slave population. Early in his speech, given on January 18, 1832, Brown recognized and admitted the evils of slavery. However, he put forth a caveat, arguing, “It might have been expected that, with so perfect a knowledge of the malady, the physician could have recommended an effectual remedy. None has yet been proposed that commands my confidence.”[19]

Essentially, Brown boiled the debate down to a question of property rights. Like many Virginians, Brown fiercely loved his homeland. He defended the centrality of slavery in Virginia’s economy in his speech to the House, posing, “Take from us these slaves without compensation, and what have we left?”[20] After all, the Brown family fortune largely hinged upon slave labor. Brown asserted, “For my own part, sir, I am not the advocate of slavery in the abstract, and if the question were upon introducing it, I should be the very last to agree to it; but I am yet to be convinced that slavery [,] as it exists in Virginia, is either criminal or immoral.”[21] The antebellum era brought about grave questions about the future of the state’s inhabitants, both white and black, and Brown’s successful political career demonstrates the ideals of one camp among the period’s many warring factions.

According to Daisy I. Read’s New London Today and Yesterday, Captain Henry Brown received a rudimentary education [22]. Read’s account is, admittedly, tainted by several inaccuracies, but if true, Brown’s later emphasis and clear esteem for education paints a heartwarming picture and says much about his personal character.

Captain Brown served on the Board of Trustees at nearby New London Academy in 1797 and 1811-1826, acting as president from 1826-1837.[23] Furthermore, his children received remarkably high-quality educations. Henry Jr. and John Thompson both attended Princeton University, the latter graduating in a three-way tie for valedictorian, and their younger brother Samuel attended West Point Academy.[24]

Captain Brown also encouraged the education of his daughters, who attended the Ann Smith Academy in Lexington, Virginia. Chartered in 1807, the school offered young women ages 13-16 lessons in “reading, writing, and arithmetic, grammar, geography, natural philosophy, chemistry, astronomy, belles lettres, French, painting, instrumental music, and embroidery” for a tuition cost of $25 per year.[25] The fact that the Brown daughters received learning beyond basic reading, writing, and domestic skills outside of private tutoring is a testament to their father’s outlook and values.