I've just finished reading Lawrence Krauss's new book Hiding in the Mirror: The Mysterious Allure of Extra Dimensions, from Plato to String Theory and Beyond, and it's very, very good. Scientifically, the book covers a lot of the same material as Lisa Randall's Warped Passages, but it's about half as long and has a wider perspective, with writing that is pithy and entertaining. Krauss's topic is not just the science of extra dimensions, but the history of various ways the idea has turned up in art and literature, and the whole question of why people find it so fascinating.
While they are ultimately concerned with the same speculative ideas about extra dimensions, Krauss and Randall's books are in many ways different. Randall is writing about her own research work, so on the one hand she is a partisan for these ideas, on the other she gets to tell the inside story of exactly how she came up with them. She goes to a lot of trouble to dig in and try and explain in as simple terms as possible the details of the physics that motivates this research, as well as exactly what it is trying to achieve, how it has evolved in recent years and where it seems to be going. Krauss also covers these topics, but is (justifiably in my view) more of a skeptic, and sets the whole story in a wider context of the long history of this kind of speculation. If you've read Randall's book, you should seriously consider reading Krauss for a different point of view. If you read Krauss and want a much more extended exposition on some of these topics, Randall is the place to go.
Krauss begins by telling the story of an episode of the Twilight Zone TV program that had quite an impact on him when he was very young. It involved a little girl who falls into another dimension and is saved by intervention of a physicist. He notes that "We all yearn to discover new realities hidden just out of sight", but that "Ultimately our continuing intellectual fascination with extra dimensions may tell us more about our own human nature than it does about the universe itself." Krauss writes about a wide range of different writers and artists who have been fascinated by the idea of extra dimensions, and some of the historical and cultural context for their work. Much of this I didn't know anything about, although his description of the science fiction short story "And He Built a Crooked House" by Robert Heinlein brought back memories of my childhood, since I had found that story very striking, but hadn't thought about it in a very long time (it involves a house based on a tessaract, a 4d version of a cube). Another interesting piece of history he unearths is that Marcel Duchamp's famous piece The Bride Stripped Bare by Her Bachelors, Even (also known as the Large Glass), was heavily influenced by ideas about projecting from four dimensions, and that Duchamp spent a lot of time trying to learn about this, including reading Poincare.
Krauss writes that, while fascinated by the idea, he himself remains a skeptic (or at least agnostic) about the actual existence of physical extra dimensions. He tells the history of attempts by theorists to use extra dimensions, from 19th century conjectures that atoms were points where a four-dimensional etherlike field leaked into three-dimensional space, to Kaluza-Klein models and the heterotic string, ending up with recent braneworld scenarios. He describes the ideas behind this recent research concisely and also explains exactly what some of the problems with these ideas are. Along the way he comes up with various obscure and interesting pieces of the history of physics I'd never heard before, for instance that in 1928 an English experimentalist named R. T. Cox found evidence of parity violation, but his results were not taken seriously.
On the topic of string theory and braneworlds, Krauss promises to be not like Fox News (i.e. actually "Fair and Balanced"), but he has truly scathing things to say, and these are a refreshing change from the unjustifiably enthusiastic depiction of these subjects that has been common in popular science writing until recently. While he is not one of the researchers actively developing some of these extra-dimensional scenarios like Randall, he has significant expertise on the subject, and was the thesis advisor of Randall's collaborator Raman Sundrum.
Krauss ends his book with an epilogue describing conversations with Gross, Wilczek and Witten about string theory. Wilczek is a skeptic, annoyed by the excessive claims made for the theory. Witten is quoted as saying that string theory "is a remarkably simple way of getting a rough draft of particle physics unified with gravity. There are, however, uncomfortably many ways to reach such a rough draft, and it is frustratingly difficult to get a second draft." He justifies work on string theory partly through progress it has led to in the understanding of strongly coupled gauge theories.
Gross is described as convinced "that the theory is simply too beautiful not to be true", an attitude that strikes Krauss "as sounding like religion more than science." With this, Krauss ends his book by quoting Hermann Weyl:
"My work always tried to unite the true and the beautiful, but when I had to choose one or the other, I usually chose the beautiful."
and concludes:
"So it is that mathematicians, poets, writers, and artists almost always choose beauty over truth. Scientists, alas, do not have this luxury, and can only hope that we do not have to make this choice."
Here, to some extent I part ways with Krauss. I don't find the 10 dimensional heterotic superstring compactified on a Calabi-Yau to be in any sense beautiful, and attempts to connect string theory with physics lead to appallingly ugly constructions, strong evidence that they are on the wrong track. Absent useful experimental results, the pursuit of compelling new mathematically beautiful insights into fundamental physics is one of the few promising ways forward. But to go down this road successfully you have to be honest about what is mathematically beautiful and what isn't.
All in all, this is by far the best book that I know of on the topic of recent speculative work on fundamental particle physics, and I strongly recommend that anyone who enjoys reading about this should get themselves a copy.