The tract of land which comprises Richmond and Gardiner was purchased in 1649 from the Abenaki Indians by Christopher Lawson. In 1719, Fort Richmond was built by Massachusetts on the western bank of the Kennebec River at what is today Richmond Village. Named for Ludovic Stewart, 1st Duke of Richmond, the fort included a blockhouse, trading post, chapel, officer's and soldiers' quarters, all surrounded by a palisade.
In 1722, Fort Richmond was attacked in a 3 hour siege by warriors from Norridgewock. Houses were burned and cattle slain, but the fort held. Brunswick and other settlements near the mouth of the Kennebec were destroyed. The defense was enlarged in 1723 during Dummer's War. On August 19, 1724, a militia of 208 soldiers departed Fort Richmond under command of Captain Jeremiah Moulton and Johnson Harmon, traveled up the Kennebec in 17 whaleboats, and sacked Norridgewock. Fort Richmond would be rebuilt in 1740, attacked by another tribe in 1750, then dismantled in 1755 when forts Shirley (also called Frankfort), Western and Halifax were built upriver.
Settled in 1725, the community was part of Bowdoinham when it was incorporated in 1762 by the Massachusetts General Court. In 1790, Revolutionary War veteran John Plummer was awarded a land grant on Plummer Road, where his son built the surviving house about 1810. But President Thomas Jefferson's Embargo of 1807 crippled the port's economy, bankrupted merchants and created a recession which lingered through the War of 1812.
The town was set off and incorporated on February 10, 1823, taking its name from the old fort. Farms produced hay and potatoes. With the arrival of steamboats in the 1830s, Richmond boomed as a shipbuilding and trade center. A brass foundry was established. The community also produced shoes, sales and wood products. Its peak years were between 1835 and 1857, endowing the town with a wealth of fine Greek Revival architecture, which today makes the old Riverport popular with tourists.