The Santa Inez Mission was not comprehended in the original plan of the padres, but nearly thirty years after the founding of the first three Missions, a colony of several families that had years before located on lands in the valley of the Santa Inez, about forty miles northwest of Santa Barbara, and beyond the mountains, appealed to the President of the Missions for the founding of one in their vicinity. They argued that they, being baptized families, were entitled to the rites of divine worship without undergoing the hardship and inconvenience of frequent trips to Santa Barbara, or La Purisima, each of which was many miles away from them.
The petition was granted, and on September 17, 1804, the new Mission of Santa Inez was founded in that valley, and dedicated to St. Agnes. One hundred and fifty persons were entered on the records, and a church was immediately started. The new colony flourished, but the earthquake of 1812 so shattered the walls of their buildings that they had to be rebuilt. The Mission prospered in flocks and herds for about fifteen years, when it appeared by one record that it had accumulated twenty-five thousand cattle, fifteen thousand sheep, and twenty-five hundred horses, and a great deal of other personal property, the flocks and herds and lands at all times constituting the basis of its wealth. The Indians in 1824 became discontented and troublesome, and many of them left the Mission. The buildings were burned and otherwise destroyed to a great extent and never fully restored. Many Indians left and never returned. The work of conversion languished, but the riches of the Mission grew in magnitude until the day of secularization.