Customer Reviews

Cost-Based Oracle Fundamentals (Expert's Voice in Oracle)

5 star:		(25)
4 star:		(1)
3 star:		(2)
2 star:		(2)
1 star:		(0)

 

 

This is the book for you.

This book is, well, in a word amazing. If you have ever been baffled or bemused by why the heck did the optimizer do that, or as Jonathan wrote on page 299:

"I am reluctant to call something a bug unless I can work out what Oracle is doing and can prove that its doing something irrational. Too many people say, Its a bug when they really mean I dont know why this happened."

You will absolutely love this book. In it you will discover the hows and whys of the optimizer. Why statistics matter, how they matter. Whats up with histograms when and where do we need them, what affect do they have.

Sprinkled throughout the book are random insights like this one:

"There are many ways to implement Oracle systems badly, and as a general rule, anything that hides useful information from the optimizer is a bad idea. One of the simple, and highly popular, strategies for doing this is to stick all of your reference data into a single table with a type column. The results can be catastrophic as far as the optimizer is concerned."

And then is goes on to say why. That is what I really really like it goes on to say why. I hate it when statements are made and no reasoning is made why. You will find none of that in this book.

Jonathan did one thing in this book that Ill definitely be stealing myself. One neat thing is every chapter ends with a list of script names and descriptions. In the text, he references these script names as well. That way, when you download the code you have a straight reference to the sample you should be running. Ive used the (extremely poor) naming convention of demo001.sql, demo002.sql and so on. Next book theyll all have names and Ill be referencing exactly like he did. Very nice.

The attention to detail, the simplicity of presentation (I dont care what level of Oracle user you are you will be able to read this book and get it). If you are advanced (ok, Ill put myself into that category), youll learn things you did not know before. If you are beginner, youll know lots more than some advanced people after reading it. The surprising thing? It isnt that hard. Well, it wasnt to me anyway maybe the math background I have helped. You do not need 10 years of experience with Oracle to get this stuff, and if you have 10 years of experience with Oracle you will get new knowledge you never had.

Im on my second scan of it re-reading things that I didnt fully absorb. What Ill be doing lots in the future is referring to it. I got the gist of everything, I know where to go when I need to explain why. Or maybe Ill just post the link to the book.

And remember, this is I of III, two more to come

There is little point to write how good this book is, since there is no other book devoted to SQL optimization exclusively. Dan Tow's book comes close, but he is focused more on a method of join graph analysis that he developed, than on details how optimizer did arrive to a certain access path. The lack of competition on the market is really surprising giving that SQL optimization is the only part of RDBMS that is justifiably complex, and would remain complex in foreseable future.

Compared to SQL optimizations all the other issues that DBA deals today look ridiculous. There is no reason why, for example export and import should be more complex than copying image file from your camera. Likewise, managing extents and segments is totally automated these days. All the manageability trend just proves a simple idea that RDBMS is nothing more than query execution engine.

Now, unlike any other RDBMS implementation area, the flow of poorly executed SQL never seems to cease. SQL Optimization is well known to be a difficult problem. Statistics information is incomplete, robust cost metrics is elusive, and the search space is explosive. The optimization goals are often conflicting. The very first idea that every SQL performance analyst discovers: "The optimization is only as good as its cost estimates". Those issues are fundamental rather than SQL DBMS vendor specific, of course. Given the scope and complexity of the problem, one citation comes to mind: "There is no emperor's way to SQL optimization".

The beauty of reading a book by a publisher not sanctioned by Oracle and by an author who doesn't work for Oracle is that they can openly mention bugs. And there are oh-so-many! This book is a superb introduction to the Cost Based Optimizer, and is not afraid to discuss it's many shortcomings. In so doing it also explains how to patch up those shortcomings by giving the CBO more information, either by creating a histogram here and there, or by using the DBMS_STATS package to insert your own statistics in those specific cases where you need to.

Another interesting thing is how this book illustrates, though
accidentally, the challenges of proprietary software systems. Much of this book and the authors time is spent reverse engineering the CBO, Oracle's bread and butter optimizing engine. Source code, and details about its inner workings are not published or available. And of course that's intentional. But what's clear page after page in this book is that for the DBA and system tuner, going about their day to day tasks, they really need inside information about what the optimizer is doing, and so this book goes on a long journal to illuminate much of what the CBO is doing, or in some cases provide very educated guesses and some speculation. In contrast, as we know and hear about often, the Open Source alternative provides free access to source code, though not necessarily to the goods themselves. What this means in a very real way is that a book like this would not need to be written for an alternative open source application, because the internal code would be a proverbial open book. That said it remains difficult to imagine how a company like Oracle might persue a more open strategy given that their bread and butter really is the secrets hidden inside their Cost Based Optimizing engine. At any rate, let's get back to Jonathan's book.

Reading this book was like reading a scientists notebook. I found it:
o of inestimable value, but sometimes difficult to sift through
o very anecdotal in nature, debugging, and constantly demonstrating that the CBO is much more faulty and prone to errors than you might imagine
o may not be easy to say I have a query of type X, and it is behaving funny, how do I lookup information on this?
o his discussion of the evolution of the product is so good I'll quote it:
"A common evolutionary path in the optimizer code seems to be the following: hidden by undocumented parameter and disabled in first release; silently enabled but not costed in second release; enabled and costed in third release."
o has excellent chapter summaries which were particularly good for sifting, and boiling down the previous pages into a few conclusions.
o it will probably be of particular value to Oracle's own CBO development teams

Some chapter highlights
-------------------

CH2 - Tablescans
explains how to gather system stats, how to use dbms_stats to set ind. stats manually, bind variables can make the CBO blind, bind variable peeking may not help, partition exchange may break global stats for table, use CPU costing when possible

CH3 - Selectivity
big problem with IN lists in 8i, fixed in 9i/10g, but still prob. with NOT IN, uses very good example of astrological signs overlapping birth months, and associated CBO cardinality problems, reminds us that the optimizer isn't actually intelligent per se, but merely a piece of software

CH4 BTree Access
cost based on depth, #leaf blocks, and clustering factor, try to use CPU costing (system statistics)

CH5 - Clustering Factor
mainly a measure of the degree of random distribution of your data, very important for costing indx scans, use dbms_stats to correct when necessary, just giving CBO better information, freelists (procID problem) + freelist groups discussion with RAC

CH6 - Selectivity Issues
there is a big problem with string selectivity, Oracle uses only first seven characters, will be even more trouble for urls all starting with "http://", and multibyte charactersets, trouble when you have db ind. apps which use string for date, use histrograms when you have problems, can use the tuning advisor for "offline optimization", Oracle uses transitive closure to transform queries to more easily opt versions, moves predicates around, sometimes runs astray

CH7 - Histograms
height balanced > 255 buckets (outside Oracle called equi-depth),
otherwise frequency histograms, don't use cursor sharing as it forces bind variables, blinds CBO, bind var peeking is only first call, Oracle doesn't use histograms much, expensive to create, use sparingly, dist queries don't pull hist from remote site, don't work well with joins, no impact if you're using bind vars, if using dbms_stats to hack certain stats be careful of rare codepaths

CH8 - Bitmap Indexes
don't stop at just one, avoid updates like the plague as can cause deadlocking, opt assumes 80% data tightly packed, 20% widely scattered

CH9 - Query Transformation
partly rule based, peeling the onion w views to understand complex queries, natural language queries often not the most efficient, therefore this transformation process has huge potential upside for Oracle in overall optimization of app code behind the scenes by db engine, always remember Oracle may rewrite your query, sometimes want to block with hints, tell CBO about uniqueness, not NULL if you know this

CH10 - Join Cardinality
makes sensible guess at best first table, continues from there,
don't hide useful information from the CBO, histograms may help with some difficult queries

CH11 - Nested Loops
fairly straightforward costing based on cardinality of each returned set multiplied together

CH12 - Hash Joins
Oracle executes as optimal (all in memory), onepass (doesn't quite fit so dumped to disk for one pass) and multipass (least attractive sort to disk), avoid scripts writing scripts in prod, best option is to use workarea_size_policy=AUTO, set pga_aggregate_target & use CPU costing

CH 13 - Sorting + Merge Joins
also uses optimal, onepass, & multipass algorithms, need more than 4x dataset size for in memory sort, 8x on 64bit system, increasing sort_area_size will incr. CPU util so on CPU bottlenecked machines sorting to disk (onepass) may improve performance, must always use ORDER BY to guarentee sorted output, Oracle may not need to sort behind the scenes, Oracle very good at avoiding sorts, again try to use workarea_size_policy=AUTO

CH 14 - 10053 Trace
reviews various ways to enable, detailed rundown of trace with comments inline, and highlights; even mentions a VOL 2 + 3 of the book is coming!

Appendix A
be careful when switching from analyze to dbms_stats, in 10g some new hist will appear w/default dbms_stats options, 10g creates job to gather stats

Conclusion
----------
I found this book to be full of gems of information that you won't find anywhere else. If you're at the more technical end of the spectrum, this is a one of a kind Oracle book and a
must-have for your collection. Keep in mind something Jonathan mentions in appendix A: "New features that improve 99% of all known queries may cripple your database because you fall into the remaining 1% of special cases". If these cases are your concern, then this book will surely prove to be one-of-a-kind for you!

Since I am the author of this book my opinion is, of course, biased. So far, Amazon.com has seen several rave reviews and a couple of poor ones. I think the critical difference between the reviews lies in one important word:

Understanding.

The purpose of the book is to show you how the cost based optimizer works. I did not write it to give you a list of quick tips that might fix a problem statement (or might make it, and dozens of others, worse). Do not buy this book if all you are after is hints and tips that you can use without thinking. If there were a few quick tricks that could fix everyone's systems, they would already be built into the optimizer code. (And the ones that can't be in the code are in the manuals)

Many people find it easier to understand a process by working through examples so that is the approach I've taken for the book; but this means that some people will find that it just doesn't suit the way they like to think. So rather than depending on reviews, you can download a sample chapter from Apress at the URL: [...] and see for yourself if the approach works for you.

You can also find a list of errors and updates at the URL: http://www.jlcomp.demon.co.uk/cbo_book/ind_book.html#addenda

I bought the book "Cost-Based Oracle Fundamentals" and I found it extremely helpful and I've been recommending to other people - both DBA's and SQL writers. I've been pretty good at tuning problem SQL in the past mostly by knowing (or learning) the data and forcing the best plan using hints. But the row estimates/cost figures that the optimizer comes up with have always been a mystery to me. This book explains a lot of what I have been seeing. (The answer is 5%!)

The book is very well laid out and it meant to conceptually show how the optimizer makes decisions and how it can be mislead into making wrong decisions. It also spells out many situations where the optimizer is making a SWAG and indicates where you should pay attention to the explain plans and perhaps hint them so they run properly.

With the suggestions in the book, I have made great strides in fixing our database so that we get rational explain plans from the optimizer. However, we have so many queries based on SYSDATE calculations that I now realize that many of the row estimates (5%) are crap and that frequently this is what has been causing us issues. (I knew that the row estimates were the issue. I just never knew where the harebrained estimates actually came from. Now I can say, "Oh, 7 million. That's 5% of 152 million rows. The optimizer doesn't know how many rows you're bringing back for SYSDATE and SYSDATE-1, so it's using 5%")

Thanks to this book, I can now also show that half the time the CBO has been picking the right plan for the wrong reasons! Who says, "Two wrongs don't make a right?" They do sometimes. Now, as I untangle all this, I have to worry about Oracle suddenly selecting the wrong plan for only one wrong reason! But I've at least been able to explain why the performance of some SQL seems to periodically go haywire and then fix itself without any changes having been made to the SQL.

I have so many more answers and explanations now that when people see the yellow and black book on my desk, they ask me, "Is this the book?" I look forward to the next two volumes in the series.

Ah, yes, the question: "Why isn't Oracle using my index?"

Have you ever asked it? Well, maybe you've never asked it, but you might have heard about someone who has... If you would like to know how to find the answer, this is the book (well, the first of three, really).

It is important to understand that the book gives you the knowledge needed to find the answer -- it doesn't try to give you a list quick-fixes or "the" answer (which of course it really couldn't). There is no "_optimizer_always_makes_correct_choice = true" hidden parameter you can set, but you can learn how to give the optimizer the best information possible. Plus you'll learn what the optimizer does with this information.

Jonathan's work is always top-drawer and this is no exception. The technical reviewers (Wolfgang Breitling and Christian Antognini) are both well-versed in the workings of Oracle in general and the optimizer in particular. I've no doubt that this helped contribute to the quality of the work. After reading this book you can't help but be more informed and possess the skills necessary to decipher why the optimizer is making the choices that it does.

First off, I have the utmost respect for Johnathan Lewis' expertise with Oracle. I loved his 8i book.

I would have left the word "Fundamentals" out of the title. It's clearly a book for intermediate to advanced study of CBO albeit a little out of date.

The major complaint I have is that Johnathan spends to much time detailing how he determined how the CBO is functioning and not enough time with:

1.) Articulating what the CBO is doing.
2.) And probably most important, what does this mean to a Developer or DBA who is using CBO in general situations (e.g. OLTP, Data Warehousing, queries, DML, star transformations, partitions etc?
3.) How could we affect CBO in a positive manner in a given situation.

These important subjects are far too frequently either not addressed, or not sufficiently addressed and buried in the gory mathematical details of how he figured out what CBO was doing.

It's almost like he is proving his theories, which is fine if you have the time to work thru them.

I don't really care how he figured it out. I trust that he has tested his theories. Perhaps he could just footnote them on a website.

I did like the way communicated Oracle feature implementation process. And some of the other insider details he derives from his analysis.

Again, his understanding of the product is second to none. His ability to communicate it in English leaves something to be desired - at least this time.

Mr.Jonathan Lewis has taken a complex subject of CBO and presented the intricate working details of this beast eloquently. I could not wait to get this book as soon as I heard about it and now I can't keep the book down!

Even after working with CBO trace so long, I am still learning a lot from this master. Mr. Lewis combats each component of CBO separately in a chapter format and provides examples & test cases for each of his observation and working principles. He anlyzes the results logically providing valuable insights. Every statement he makes is verifiable using the test case he provides or metalink note is referenced and no myth producing elements in his statements.

Resolving the root cause solves the problem once for all. Understanding the optimizer internals is very important to resolve the root cause of a SQL performance problem. [You can add a hint and resolve the problem, but that is usually a band-aid, not a solution to the problem]. There is no other book that deals with the root cause analysis of CBO comparable to this book.

He goes further explaining the reasoning behind the optimizer decisions. While reading the book, I had a few flashbacks such as ' Aha, that's why that query didn't work before rewriting it! I wish I read this book earlier and could have solved without a time-consuming rewrite'.

Every DBAs / performance tuners should read this book and hold on to it.

Kudos to Mr. Lewis! Looking forward for more books!!

This is the book that one definitely needs to read (and understand) in order to get an idea what the CBO is all about.

Jonathan's examples and presentation is excellent. Though the topic is quite intense, the book is quite pleasant to read. This book is a worthy follow up the classic "Practical Oracle8i".

As regards the usefulness of the book in real world scenarios, I can only say that if one wants to approach SQL tuning as a science, one needs to learn the CBO, and this book is essential.

This is volume 1 or 3. I will definitely be eagerly waiting for the next volumes.

This book gets a prized spot on my technical shelf. It is a challenging read, and has taken the most time of any book I have to get through, but it is quite good. The optimizer is such a core part of what makes Oracle worth the investment, that any DBA who does not understand it is doing a disservice to the company investing such capital into the product.

Lewis' use of proof through example is second to none, and leaves me feeling confident about the knowledge I've gained. His insights are not only theoretical, but quite applicable. I learned much about manipulating and using the DBMS_STATS package, about data model design and why data knowledge is so critical to the DBA.

My nervousness about hints and about upgrades was reinforced with concrete examples that have improved my test plans, upgrade methodologies and overall made me a more competent DBA. I've directly used this knowledge to support my clients in better ways than I would have before it, and that is well worth the price of admission and the time to get through such an in-depth work.

Thanks Jonathan!