When Calvin Seibert was a boy, he and his two brothers treated the nascent town of Vail like their playground. Their father, Pete Seibert, had cofounded the resort in 1962, and the boys spent their days running amok through the newly built village—“a free-form maze where risks could be taken and games invented,” he recalls.

Then a Bridge Street water-balloon fight got out of hand in 1968, and someone told them to take their shenanigans elsewhere. Seibert found a new pastime: building makeshift structures in his parents’ yard. His father showed him a photo of a brutalist French building designed by the famed modernist Marcel Breuer. Shortly thereafter, his mother had some sand delivered and bought bags of cement at the lumber yard in Minturn, and Seibert set about building his own version of the concrete fortress. It was the beginning of a craft that recently has brought him improbable renown, as a mountain man-turned-sandcastle master in New York City. “A lot of what I made had the quality of abstract sculpture, though it was years before I came to understand that,” Seibert, now 60, says of his childhood hobby.

His family assumed he would become an architect, but since graduating from the School of Visual Arts in New York in 1983, Seibert has worked mostly as an artist’s assistant, with stints as a house painter, carpenter, and art installer. He reserves his summers for sandcastles, making sketches all winter, then letting his imagination take over on the beach. “When I’m building a sandcastle, I want to be surprised,” he says. “I start out with a general idea that feels different from whatever I did the day before, and then just let things take their course.”

He builds four days a week, usually at Rockaway Beach on Long Island—where his commute on the A Train takes an hour and a half each way—but also at Jones Beach, where his castles remain untouched for days. Each castle takes shape with the help of handmade plastic trowels, putty knives, and a five-gallon bucket to move water and sand. “It’s physically demanding,” he says. “I can spend as much as 10 hours working on a single castle.”

After he was profiled on the CBS Evening News (and in Architectural Digest, the New York Times, and Smithsonian magazine), the world has taken notice. Aside from the rare commission—in April, Seibert traveled to the Yucatan Peninsula to teach a workshop at Powder Mountain’s Summit Series, and has sculpted sand in France for Hermès—he works for free. On Instagram, where he posts photos of his creations, his handle (@calvinseibert) counts more than 10,300 followers. “This means I don’t have internet at home, or a car, or a dog, or air conditioning,” he says. “And I hardly ever go to the movies or eat out.”

As the ski resort founder’s son, Seibert holds a lifetime ski pass to Vail, and he still makes it home to ski a day or two each year (his favorite run is Avanti). But he mostly whiles away his winters in New York City, dreaming about next summer’s castles.

“It’s a far cry from the landlocked sandscape of my childhood,” Seibert says. “And yet, 50 years on, nothing really has changed.”