James Jackson was born on October 25, 1782 to James and Mary Jackson in County Monaghan, Ireland. He grew up in Ireland until he moved to the U.S. in 1799; first to Pennsylvania before later moving to Tennessee. In Tennessee he met his wife Sarah Moore McCulloch who he would marry in 1810. They had ten children together.
James later moved to the area that would be come the Florence/Lauderdale area. Here he became part of the seven trustees of the Cypress Land Company. The Cypress Land Company was created under the pretense that Florence would become a major commercial town due to its connection to the Tennessee River. This was backed by President Andrew Jackson. The company would result in great success for both the people invloved and the city of Florence. It would also go on to aid in the creation of TVA and thus the Wilson Dam.