In this paper, an inversion algorithm for the retrieval of total density, pressure, and temperature in the Earth's lower atmosphere from refraction angle measurements by a visible-light imager is presented. A detailed description of the retrieval algorithm is provided, and its performance under a variety of conditions is assessed, including a rigorous evaluation of the statistical and systematic uncertainties. Tests show that visible-light images of stellar refraction can provide reasonably accurate measurements of the atmospheric bulk properties from the surface (or cloud top heights) to roughly 30 km altitude. The combined statistical and systematic uncertainties in density and temperature profiles retrieved with a vertical resolution of 1 km decrease from 3.1% and 5 K, respectively, at 30 km to 0.8% and 3.8 K at 20 km and 0.35% and 1.5 K at 10 km for an imager with a sampling rate of 10 Hz and pointing control/knowledge of 0.002° (1σ). This level of accuracy and resolution is adequate for a combined extinctive/refractive occultation technique for remotely sensing the Earth's lower atmosphere that has recently been demonstrated using data taken by the Ultraviolet and Visible Imagers and Spectrographic Imagers (UVISI) on the Midcourse Space Experiment (MSX) spacecraft.