Tony Smith was born in 1912, in South Orange, New Jersey. As a child he suffered from tuberculosis, and to prevent the disease from spreading to his six siblings, he was quarantined in a one-room structure behind his parent's house and was cared for by a private nurse. He spent time there building models from small medicine boxes.

During the Depression, Smith attended college briefly, but returned home to help run the family's tool and die manufacturing business. He also took painting and drawing classes at the Art Student's League in New York, and in 1937, moved to Chicago to study architecture. Soon, he became an apprentice to Frank Lloyd Wright. Smith supported himself as an architect for more than twenty years.

After a car wreck in 1961 left him with a blood disease that kept him from driving to building sites, Smith gave up architecture, and began teaching design at several colleges in New York.

Smith's first solo exhibition took place in 1966 at the Wadsworth Athenaeum in Hartford, Connecticut. His work can be seen at the National Gallery of Art Sculpture Garden in Washington, D.C.; Olympic Sculpture Park in Seattle, Washington; the Rijksmuseum Kroller-Muller, Otterlo, Netherlands; and the Shizuoka Prefectural Museum of Art in Shizuoka, Japan.