Is the Whitney Museum’s new, ultra-touristy location just off the High Line already going soft? It looks like the answer may be yes, at least when it comes to public-facing installations.

Calvin Tomkins, a 2011 Whitney Gala honoree, took aim at the venerable American institution in his latest New Yorker profile on the artist Charles Ray. In the piece, Tomkins mentions that a sculpture commissioned from artist for a plaza outside the museum was rejected on the grounds that today’s Meatpacking District crowd might find it too controversial. The nine-foot-tall sculpture, called Huck and Jim, features exposed male genitalia.

Ray received the commission in 2009. The Whitney’s identity as an institution dedicated to American art inspired the artist to consider Mark Twain’s classic, and controversial, novel, The Adventures of Huckleberry Finn. He showed the preliminary designs to director Adam Weinberg and chief curator Donna De Salvo, both of whom reportedly seemed pleased with the project.

The article reveals that the concerns came later, and stemmed from notions that a sculpture of a nude African-American man next to a nude white youth would be too challenging for those who would pass by the public space without understanding the context. Which could turn them off from entering the museum. In 2010, Weinberg reportedly told Ray he would be happy to install the sculpture somewhere else on the museum’s property, but that it could not be placed in the public-facing plaza.

Ray rejected the idea.

“I don’t want whatever becomes of it to be less than the original idea, and the original idea was for it to be there,” he told Tomkins. “Listen, I’m not naïve to the controversies this would generate—I told them that controversies would be a forest we had to navigate through.”

He continued to work on the sculpture, which is now on display as part of his retrospective at the Art Institute of Chicago, titled, “Charles Ray: Sculpture, 1997–2014.” However, the Institute has kept in line with the disclaimer trend in recent years. The website states that “some works in this exhibition may not be suitable for younger viewers.”

This isn’t the first time public art has been jettisoned, although some projects have been able to be pushed through, depending on the artist’s will, and star power. In 2012, for example, Damien Hirst’s giant bronze statue of pregnant woman with a vivisected belly revealing the fetus was placed in the sleepy seaside town of Ilfracombe, Devon, to the annoyance of many locals. BBC News interviewed residents about the 66-foot-fall statue, named Verity. One woman said, “It’s just not nice. I think it will frighten a lot of children.”

As Ray told Tomkins, “I’m over the fact that Huck and Jim is not going to be at the Whitney, and I understand the reasons.”