Why does the Oracle cbo interest me so much?

There is a short answer and a very long answer to this.  I will try and keep the answer somewhere in the middle of short and long.

The first assumption we must accept and from my observations over the past several years, most Oracle developers don’t understand the Cost Based Optimizer, yet they use it everyday.

Way back in the dark ages, full table scans were evil and we used hints to get rid of this dreaded access path.  As you can see from the bellow query, we got a full table scan of t1

So we solve this by adding a hint.

That’s better, we think but lets look at a few things.  Did the hint really help? The first query had 70,308 consistent gets, 3,884 physical reads and cost was 1,065.  The second query where we thought we were being cleaver had 1,057,564 consistent gets, 5868 physical reads and the cost was 990K.

Well, maybe I was not so cleaver by forcing Oracle to use an index, maybe the CBO is smart enough to put together the correct plan and the full table scan is not evil after all.

A few years ago I was at ODTUG and heard Maria Colgan speaking about the CBO and I was hooked.  Maria made me realize I need to understand what the CBO was doing.  Yes sometimes the CBO gets it wrong, but there are ways to correct that.

I’m now putting together a series of presentations on understanding the CBO using the scientific approach.  Why not just publish how the CBO works? Why not publish ROT (Rule Of Thumb) for the CBO.  Well the answer to that is quite simple.  Every environment is different and it’s much better to understand your data and how the CBO works with your data and queries.  There will be a presentations that will go through how to read an explain plan, however the other presentations will teach you how to establish what the CBO is doing in your environment and you will be able to explain it.  That makes you smarter and more valuable.

Stay tuned, I will be posting blurbs and hints on the CBO.

Oracle 10053 trace viewer

There are two trace files that I use when tuning.  Cary Millsap @CaryMillsap and Method-R @MethodR have a great tool for analyzing the 10046 trace file. There is the profiler and my personal favorite mrskew http://method-r.com/store .

But the 10053 trace file is cumbersome to read and interpret.  Jonathan Lewis @JLOracle posted this 10053 trace file viewer on oaktable.net. http://www.oaktable.net/contribute/10053-viewer .

If you are an Oracle DBA, Developer or Performance Engineer, these should be in your tool box.

Exploring the CBO

Lets start with a simple experiment on what the CBO can do with query optimization. We are going to build on the structure as we go to see if we can help out the CBO with more information.

Test 1) we will start with two tables bigtable and owners and see what plan the CBO comes up with.

Test 2) we will analyze the tables and see what plan the CBO comes up with.

Test 3) we will add primary key and foreign key constraints and see what plan the CBO comes up with.

Test 4) we will add an index on the foreign key constraint and see what plan the CBO comes up with.

 

We know that these two queries are equivalent. So, can the CBO get from query A to query B

A

B

SELECT COUNT(*)

FROM OWNERS O,

BIGTABLE BT

WHERE O.USERNAME = BT.OWNER

and o.username like ‘RL%’;

SELECT COUNT(*)

FROM BIGTABLE BT

WHERE BT.OWNER LIKE ‘RL%’;

 

 

The first pass we are going to use bigtable and owners with no statistics, no indexes and no primary key foreign key restraints.

drop table bigtable;

drop table owners;

create table bigtable as select * from dba_objects;

create table owners as select * from dba_users;

 

alter session set tracefile_identifier = owi_trace;

— This line will generate a trace file of what the CBO did.

alter session set events ‘10053 trace name context forever, level 1’;

 

 

SELECT COUNT(*)

FROM OWNERS O,

BIGTABLE BT

WHERE O.USERNAME = BT.OWNER

and o.username like ‘RL%’;

 

What did the CBO come up with?

 

============

Plan Table

============

—————————————+———————————–+

| Id | Operation | Name | Rows | Bytes | Cost | Time |

—————————————+———————————–+

| 0 | SELECT STATEMENT | | | | 437 | |

| 1 | SORT AGGREGATE | | 1 | 15 | | |

| 2 | HASH JOIN | | 62 | 930 | 437 | 00:00:06 |

| 3 | TABLE ACCESS FULL | OWNERS | 1 | 9 | 3 | 00:00:01 |

| 4 | TABLE ACCESS FULL | BIGTABLE| 2398 | 14K | 434 | 00:00:06 |

—————————————+———————————–+

Predicate Information:

———————-

2 – access(“O”.”USERNAME”=”BT”.”OWNER”)

3 – filter(“O”.”USERNAME” LIKE ‘RL%’)

4 – filter(“BT”.”OWNER” LIKE ‘RL%’)

This is intresting, the CBO decided to apply two filters.

 

Well this is not the best execution plan, so lets see what happens when we analyze the tables.

 

analyze table bigtable compute statistics;

analyze table owners compute statistics;

 

alter session set tracefile_identifier = owi_trace;

alter session set events ‘10053 trace name context forever, level 1’;

 

 

SELECT COUNT(*)

FROM OWNERS O,

BIGTABLE BT

WHERE O.USERNAME = BT.OWNER

and o.username like ‘RL%’;

============

Plan Table

============

—————————————+———————————–+

| Id | Operation | Name | Rows | Bytes | Cost | Time |

—————————————+———————————–+

| 0 | SELECT STATEMENT | | | | 437 | |

| 1 | SORT AGGREGATE | | 1 | 13 | | |

| 2 | HASH JOIN | | 62 | 806 | 437 | 00:00:06 |

| 3 | TABLE ACCESS FULL | OWNERS | 1 | 8 | 3 | 00:00:01 |

| 4 | TABLE ACCESS FULL | BIGTABLE| 2398 | 12K | 434 | 00:00:06 |

—————————————+———————————–+

Predicate Information:

———————-

2 – access(“O”.”USERNAME”=”BT”.”OWNER”)

3 – filter(“O”.”USERNAME” LIKE ‘RL%’)

4 – filter(“BT”.”OWNER” LIKE ‘RL%’)

 

That was not much help. Lets see what happens when we add in primary key, foreign key restraints.

 

alter table bigtable add constraint bigtable_pk primary key (object_id);

alter table owners add constraint owners_pk primary key (username);

alter table bigtable add constraint bigtable_fk foreign key (owner) references owners(username);

 

alter session set tracefile_identifier = owi_trace;

alter session set events ‘10053 trace name context forever, level 1’;

 

 

SELECT COUNT(*)

FROM OWNERS O,

BIGTABLE BT

WHERE O.USERNAME = BT.OWNER

and o.username like ‘RL%’;

============

Plan Table

============

————————————–+———————————–+

| Id | Operation | Name | Rows | Bytes | Cost | Time |

————————————–+———————————–+

| 0 | SELECT STATEMENT | | | | 434 | |

| 1 | SORT AGGREGATE | | 1 | 5 | | |

| 2 | TABLE ACCESS FULL | BIGTABLE| 2398 | 12K | 434 | 00:00:06 |

————————————–+———————————–+

————————————–+———————————–+

Predicate Information:

———————-

2 – filter((“BT”.”OWNER” LIKE ‘RL%’ AND “BT”.”OWNER” IS NOT NULL))

Well this is better. By adding the foreign key constraint, the CBO eliminated the owners table. But we can do better. Let’s add a index on bigtable.owner and see what happens.

 

create index bigtable_idx1 on bigtable(owner);

analyze index bigtable_idx1 compute statistics;

alter session set tracefile_identifier = owi_trace;

alter session set events ‘10053 trace name context forever, level 1’;

 

 

SELECT COUNT(*)

FROM OWNERS O,

BIGTABLE BT

WHERE O.USERNAME = BT.OWNER

and o.username like ‘RL%’;

============
Plan Table
============
------------------------------------------+-----------------------------------+
| Id | Operation | Name | Rows | Bytes | Cost | Time |
------------------------------------------+-----------------------------------+
| 0 | SELECT STATEMENT | | | | 7 | |
| 1 | SORT AGGREGATE | | 1 | 5 | | |
| 2 | INDEX RANGE SCAN | BIGTABLE_IDX1| 2398 | 12K | 7 | 00:00:01 |
------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
2 - access("BT"."OWNER" LIKE 'RL%')
2 - filter(("BT"."OWNER" LIKE 'RL%' AND "BT"."OWNER" IS NOT NULL))

 

Now lets see what the CBO did to get here.
SQL:******* UNPARSED QUERY IS *******

SELECT COUNT(*) “COUNT(*)” FROM “OPS$ORACLE”.”OWNERS” “O”,”OPS$ORACLE”.”BIGTABLE” “BT” WHERE “BT”.”OWNER”=”O”.”USERNAME” AND “O”.”USERNAME” LIKE ‘RL%’

JE: eliminate table: OWNERS (O)

JE: Replaced column: O.USERNAME with column: BT.OWNER

Registered qb: SEL$84E079A4 0xadec2578 (JOIN REMOVED FROM QUERY BLOCK SEL$1; SEL$1; “O”@”SEL$1”)

**************************

Predicate Move-Around (PM)

**************************

PM: PM bypassed: Outer query contains no views.

PM: PM bypassed: Outer query contains no views.

query block SEL$84E079A4 (#0) unchanged

FPD: Considering simple filter push in query block SEL$84E079A4 (#0)

“BT”.”OWNER” IS NOT NULL AND “BT”.”OWNER” LIKE ‘RL%’

try to generate transitive predicate from check constraints for query block SEL$84E079A4 (#0)

finally: “BT”.”OWNER” IS NOT NULL AND “BT”.”OWNER” LIKE ‘RL%’

Final query after transformations:******* UNPARSED QUERY IS *******

SELECT COUNT(*) “COUNT(*)” FROM “OPS$ORACLE”.”BIGTABLE” “BT” WHERE “BT”.”OWNER” IS NOT NULL AND “BT”.”OWNER” LIKE ‘RL%’

Well that’s special. The CBO came up with a query that is close to query B.

 

B

C

SELECT COUNT(*)

FROM BIGTABLE BT

WHERE BT.OWNER ‘RL%’;

SELECT COUNT(*) “COUNT(*)” FROM “OPS$ORACLE”.”BIGTABLE” “BT” WHERE “BT”.”OWNER” IS NOT NULL AND “BT”.”OWNER” LIKE ‘RL%’
The only real difference between query B and query C is the CBO added bt.owner is not null. This is curious because if we are looking for where owner like ‘RL%’ then by definition owner would be not null.
What did we learn?

1) When we only have tables with no supporting structures the CBO selected full table scans on the two tables then did a hash join.

2) When we analyzed the tables to get good statistics, the CBO still did full table scans and a hash join.

3) When we added in the primary key foreign key constraints, the CBO knew that if bigtable.owner existed then there was a matching row in owners.username. Because of this, the CBO was able to eliminate the join condition bt.owner = o.username and eliminate the owners table from the query. This got us a full table scan on bigtable.

4) When we added the index on bigtable.owner the CBO decided an index range scan on bigtable_idx1 would return the correct results. This is because all the information needed to satisfy the query.