Are you saying that true art needs to be immediately understandable without requiring some background knowledge? This is not true for e.g renaissance art either. It usually uses a host of obscure (to modern people) symbols to communicate.
Obscurity, as a general rule, is not a virtue in itself.
Consider books of Stanisław Lem and Gene Wolfe. You could say both are difficult to read science-fiction. Both use sophisticated language and long, intricate sentences. After reading several books by each author, I came to this conclusion:
- Lem's books are difficult, but you're getting something out of them. They are thought-provoking, make you see the world or things they refer to in a different light, or contain things no one thought(or described in a book) before.
- Gene Wolfe's "Book of the New Sun"((I read all of it and it wasn't my first Wolfe's book) is difficult for the sake of being difficult. They are references you are not going to get if you are not an euridite. The book is designed for erudites. It's still enjoyable to a point and the vision of the world is interesting. Yet I can't help but notice that almost all references in that book are hollow. They don't serve any purpose. Many characters in the book are named after saints. So what ? Nothing. Some stories characters tell to each other resemble other stories, from Greek myths and whatnot. So what ? Nothing. It's just a reference. The world is thousands years old, and it may be a distorted story from the past. Or maybe not. A quote from wikipedia: "for example, a backyard full of morning glories is an intentional foreshadowing of events in Free Live Free, but is only apparent to a reader with a horticultural background".
Having a feeling that I didn't catch everything I went to several message boards dedicated to Gene Wolfe's books. Surely they can explain what's so great about these books. But they fail to convince me. They keep blabbering about how Severian is an unreliable narrator (there are a few instances of this, but they're not easy to spot). So what ? Nothing. It doesn't change anything in the book, at best it adds another "what if". Severian is a slimy character willing to lie if it benefits him. Move on.
Gene Wolfe's books are huge collections of references which make reader feel warm and fuzzy inside for getting them. If you meet someone who likes his books, it's because they make him feel special and superior. Not because of a great story, action, thought provoking ideas or very memorable characters. Additionally, the books are a big wildcard. They are deliberately kept very vague and appeal to conspiracy theorists. They could be interpreted in any way you wish.
So there you have it - background knowledge. It can be very easily used - and often is - to woo the public because it makes them feel special and superior. When you explain it away, does it make the work more interesting (like Obfuscated C) or you don't care (like Gene Wolfe's morning glories) ?
I think what you're describing here is more a problem with Gene Wolfe message boards than with Gene Wolfe's writing. They (or at least the ones I've run into) seem to delight in exploring the super-obscure details that show how clever they are, while ignoring the broad strokes which are the meat of the stories. Wolfe's books are these great symphonies working on multiple levels, and these boards focus on finding the Easter eggs.
I'm not going to tell you you're wrong not to like Wolfe. His is a very particular style, and I know people whose literary judgement is at least as good as mine who do not like it.
For my taste, though, taking an intriguing setting, telling a vivid story in it, and not spelling out all the answers is about as thought-provoking as literature comes. I won't pretend I can tell you exactly what happens at the end of the Wizard Knight or the Soldier series, but I can tell you that I've deeply enjoyed rereading them, and plan to continue doing so every few years for the rest of my life, and the mysteries make it more appealing to me, not less.
I wouldn't say I don't like it - but I think it's overblown. The Book of the New Sun has very interesting atmosphere and bits with a lot of flavor (alzabo, the introduction of Dorcas). Plus he actually managed to paint a convincing image of a very, very, very old world where you can't even dig in the ground without finding some bits of history. And my interpretation is that "Urth" is actually an evolved name of "Earth". I also read a couple of stories in similar way. It was quite amusing that in a play featuring the creation of Earth, an autarch appeared on day 1. It's almost as if the society can't imagine a world without one.
But the writing is really dry, some parts longer than they need to be, and majority of characters not memorable. I liked Jonas and the encounter with the mad autarch.
Overall, I would rate it 3.5-4 out of 5. It would be a great read if it was a bit more condensed.
As for the boards, they reminded me of the hunt for the fifth replicant among Blade Runner fans. The script was changed and the scene with one replicant was removed, but dialogue still mentioned 5 replicants. The speculations were batshit insane, until Ridley Scott simply said it was an error in the script.
Wolfe does have interesting ideas sometimes. The first book about Latro was fun in a perverse way. A character who completely forgets what he saw yesterday - yet he always found something new to say about his companions. They are shown in different light as they travel. It sure beats Terry Pratchett introducing the Librarian in exactly the same way for the 20th time. At one point the necromancer (in the classical sense) casts a spell and permanently changes to a woman. Aside from a journal entry for one day, Latro never notices. And how could he ?
Well, by saying you don't like Wolfe you are just stating a personal preference. The fact that you have found a very specific reason for your dislike doesn't any in way invalidate Wolfe's authorship or his readers as much as it further illuminates your own way of thinking.
In fact, in one sense your explanation is painfully ironic, as you seem to consider yourself morally superior to those readers who -- you claim -- read Wolfe for entirely superficial reasons. Are we really stooping so low now as to criticize people for their reading tastes or to second-guess their reasons for reading what they do?
Personally I derive as much pleasure from reading Wolfe as I do from Lem, although I would never compare them. And the key word is "pleasure": I, like I suppose most people, read books to enjoy them. I enjoy Wolfe for many reasons, but "moreal superiority" is not one of them. And I am wary of terms such as "literary significance" to guide my tastes; that's a term for scholars and academics to worry about, as a reader I am concerned with books and stories that affect me emotionally and stimulate me intellectually.
I see your point about Wolfe's complexity being more of a device and less about genuine depth. Despite this, I do enjoy the richness it imparts on the narrative. Wolfe's template is Borges, who uses similar devices and whose stories often have a sort of insular quality where the narrative only exists to create a clever gadget whose cleverness can be admired, but cannot be applied to anything outside itself. For example, Borges has a neat story, The House of Asterion, about a lonely person who roams a large house with many corridors; at the end of the story is killed, and we realize that he's the minotaur in the myth about Theseus.
As for The New Sun, I think that Severian's unreliability is more a ruse (or at best evidence of a flawed character) than an important plot point -- compare this to his Latro trilogy, set in ancient Greece, where the main character, having suffered brain damage, is unable to form new memories, and is therefore genuinely unreliable. The Short Sun also has a character who loses his sense of identity when his mind is merged with another's, and his lack of reliability comes from an inability or refusal to recognize who he actually is.
(Incidentally, while I admire The New Sun greatly, I found its two loose sequels, The Long Sun and The Short Sun, to be much more emotionally stimulating. It helps that the main characters are not borderline sociopaths. They are also less prone to the kind of cryptic connections that are evident in the first book.)