If you use tax dollars to literally carve the foundation of religion into a public building, you are conducting religion. Tacit endorsement.
Thing is about prayer in school...people shouldn't have to sit there while a public institution that they have exactly as much claim to uses time and or resources on religion, or a religion they want nothing to do with. When I go to a public school, I shouldn't have to deal with any form of Hinduism any more than a Hindu should have to deal with any form of christianity. We both have equal rights there. Its a public place. If I'm an atheist, I shouldn't have to sit there and have my time wasted on whatever religious thing they're doing.
By having this stuff mixed up with a public institution, there is a tacit endorsement, and social pressures or norms come into play. The only way to defeat that sort of thing is to eliminate it completely, so there is zero time spent on anything that can be negative for anyone with regards to religion. This does not carry over to geology, history, biology, or physics however, regardless of how well those things don't play into (insert religious belief). If people wanna believe the world is flat they can do it on their own time. Thats perfectly fine. Public institutions aren't the place to spread the word for any particular faith.
"Sit there are deal with it" isn't any more acceptable when it comes to freedom from religion than anything else in the personal rights sphere.