Colton was educated at Eton and King's College, graduating with a B.A. in 1801 and an M.A. in 1804. In 1801 he was presented by the college with the perpetual curacy of Tiverton's Prior's Quarter in Devon, where he lived for many years. He was appointed to the vicarage of Kew and Petersham in 1812. His performance of church-related functions at both locations was erratic: at times conscientious and brilliant while at other times cursory and indulgent. He left formal church service, and England, in 1828. Contemporaries believed that he had fled from his creditors, who took out a legal "docket" against him, identifying him as a wine-merchant.
For two years Colton traveled throughout the United States. He later established a modest residence in Paris. There he invested in an art gallery and had a large private collection of valuable paintings. Other pastimes included wine collecting and partridge-shooting. He also frequented the gaming salons of the "Palais Royal" and was so successful that in a year or two he acquired the equivalent of 25,000 English pounds. He continued gambling, however, and lost his French fortune. At the time of his death, Colton was living on funds received from his immediate family. An illness required surgery, but Colton dreaded the operation. He eventually killed himself rather than undergo the procedure.
Colton's books, including collections of epigrammatic aphorisms and short essays on conduct, though now almost forgotten, had a phenomenal popularity in their day. Toward the end of 1820, Colton published Lacon, or Many Things in Few Words, addressed to those who think., in a small cheap edition. It attracted attention and praise, however, and five additional printings were issued in 1821. Lacon, Vol. II appeared in 1822. In 1822 Colton re-published a previous work on Napoleon, with extensive additions, under the title of The Conflagration of Moscow. In Paris he printed An Ode on the Death of Lord Byron for private circulation and continued to write. At his death he left an unpublished poem of 600 lines called Modern Antiquity.