Sunday, December 30, 2012

Invokers rights

If synonyms are not an option, how can invoker rights be implemented to allow users access to stored PL/SQL code?

The sample configuration is as follows:
- User app1, contains application tables and code
- User firefight will have update privileges on the tables and execute on all stored code
- User read_only has read only (select) privileges on the tables and only has execute on stored code which does not update

The problems are:
- What if there is more than one application owner with duplicate table names? Synonyms are problematic for firefight user because the correct schema needs to be established prior to update particularly when using stored PL/SQL
- The read_only user obviously cannot have execute privileges to a package which provides update privileges. In essence this means that the user cannot see these packages at all.

Create demo users:
 CREATE USER app1 IDENTIFIED BY app1;  
 CREATE USER read_only IDENTIFIED BY read_only;  
 CREATE USER firefight IDENTIFIED BY firefight;  
 GRANT CREATE SESSION TO app1, read_only, firefight;  
 GRANT CREATE TABLE, CREATE PROCEDURE, CREATE SEQUENCE TO app1;  
 GRANT UNLIMITED TABLESPACE TO app1;  

Connect as app1 and create a table:
 CREATE TABLE t1(artist_id   NUMBER,   
                 artist_name VARCHAR2(50 CHAR),   
                 CONSTRAINT t1_pk PRIMARY KEY (artist_id));  
 CREATE SEQUENCE artist_id_seq START WITH 1 INCREMENT BY 1;  

 GRANT SELECT ON t1 TO read_only;  
 GRANT SELECT, INSERT, UPDATE, DELETE ON t1 TO firefight;
 GRANT SELECT ON artist_id_seq TO read_only, firefight;

Create a simple API allowing update and insert privileges and populate the table with some sample rows.
 CREATE OR REPLACE PACKAGE t1_api  
 AS  
  PROCEDURE do_insert(pi_artist_name IN t1.artist_name%TYPE);  
  PROCEDURE do_update(pi_artist_id  IN t1.artist_id%TYPE,   
            pi_artist_name IN t1.artist_name%TYPE);  
 END t1_api;  
 /  
 show errors  
 -- By default uses DEFINER privileges  
 CREATE OR REPLACE PACKAGE BODY t1_api  
 AS  
  PROCEDURE do_insert(pi_artist_name IN t1.artist_name%TYPE)  
  IS  
  BEGIN  
   INSERT INTO t1(artist_id, artist_name)  
   VALUES(artist_id_seq.nextval, pi_artist_name);  
   COMMIT;  
  END do_insert;  
  PROCEDURE do_update(pi_artist_id  IN t1.artist_id%TYPE,   
            pi_artist_name IN t1.artist_name%TYPE)  
  IS  
  BEGIN  
   UPDATE t1  
   SET  artist_name = pi_artist_name  
   WHERE artist_id = pi_artist_id;  
   COMMIT;  
  END do_update;  
 END t1_api;  
 /  
 show errors

 exec t1_api.do_insert('Pink Floyd');  
 exec t1_api.do_insert('The Smiths');  
 exec t1_api.do_insert('The Beatles');  
 COMMIT;  

Given the scenario above, read_only cannot be given any privileges on the package at all otherwise they will be able to update the contents of the table. This results in the following error:
 GRANT EXECUTE ON t1_api TO firefight;

 CONNECT read_only/read_only;  
 exec app1.t1_api.do_insert('REM');  
 /* Returns an expected error:
 Error starting at line 40 in command:
 exec app1.t1_api.do_insert('REM')
 Error report:
 ORA-06550: line 1, column 7:
 PLS-00201: identifier 'APP1.T1_API' must be declared
 ORA-06550: line 1, column 7:
 PL/SQL: Statement ignored
 06550. 00000 -  "line %s, column %s:\n%s"
 *Cause:    Usually a PL/SQL compilation error.
 *Action:
 */

 CONNECT firefight/firefight;  
 exec app1.t1_api.do_insert('Foo Fighters');  
 -- Works successfully, "anonymous block completed"

 -- Extra row can be seen in the table
 SELECT *  
 FROM  app1.t1  
 ORDER BY artist_id;

 ARTIST_ID ARTIST_NAME                                      
 ---------- --------------------------------------------------
          1 Pink Floyd                                         
          2 The Smiths                                         
          3 The Beatles                                        
          4 Foo Fighters                                       
Although this works, read_only is prevented from viewing the contents of the package - ideally, for support purposes, read_only should be able to view the contents of the packaged code. Invoker rights is ideal for this scenario.

So the next step is to amend the package to use invoker rights. Since a synonym is not an option in this case, the schema owner is prefixed to the name of the table to ensure the correct table is updated. Note that this also has the advantage of working throughout the development life cycle, as the code is moved to different environments, the application owner schema will always be updated.

Firstly, amend the calling SQL script to use variables.
 set define on verify on  
 DROP PACKAGE t1_api;
 COLUMN the_user new_value app_user;  
 SELECT user AS the_user  
 FROM  dual;  
 -- Pass the value of the compiling schema (app1) into the script to compile the stored procedure
 START t1_api_invoker.sql '&app_user' 

Now update the package to use the variable. Note the use of the double "." to ensure the object names are prefixed by the schema. &&1 references the value of the first argument passed into the script - &app_user.
 set define on verify on  
 CREATE OR REPLACE PACKAGE t1_api  
 AUTHID CURRENT_USER  
 AS  
  PROCEDURE do_insert(pi_artist_name IN t1.artist_name%TYPE);  
  PROCEDURE do_update(pi_artist_id  IN t1.artist_id%TYPE,   
            pi_artist_name IN t1.artist_name%TYPE);  
 END t1_api;  
 /  
 show errors  
 -- By default uses DEFINER privileges  
 CREATE OR REPLACE PACKAGE BODY t1_api  
 AS  
  PROCEDURE do_insert(pi_artist_name IN t1.artist_name%TYPE)  
  IS  
  BEGIN  
   INSERT INTO &&1..t1(artist_id, artist_name)  
   VALUES(&&1..artist_id_seq.nextval, pi_artist_name);  
   COMMIT;  
  END do_insert;  
  PROCEDURE do_update(pi_artist_id  IN t1.artist_id%TYPE,   
            pi_artist_name IN t1.artist_name%TYPE)  
  IS  
  BEGIN  
   UPDATE &&1..t1  
   SET  artist_name = pi_artist_name  
   WHERE artist_id = pi_artist_id;  
   COMMIT;  
  END do_update;  
 END t1_api;  
 /  
 show errors  

 -- Note that execute is now granted to read_only and firefight
 GRANT EXECUTE ON t1_api TO firefight;
 GRANT EXECUTE ON t1_api TO read_only;

Connect as read_only and run an update.
 > CONNECT read_only/read_only  
 Connected  
 > exec app1.t1_api.do_insert('REM')  
 Error starting at line 73 in command:  
 exec app1.t1_api.do_insert('REM')  
 Error report:  
 ORA-01031: insufficient privileges  
 ORA-06512: at "APP1.T1_API", line 6  
 ORA-06512: at line 1  
 01031. 00000 - "insufficient privileges"  
 *Cause:  An attempt was made to change the current username or password  
       without the appropriate privilege. This error also occurs if  
       attempting to install a database without the necessary operating  
       system privileges.  
       When Trusted Oracle is configure in DBMS MAC, this error may occur  
       if the user was granted the necessary privilege at a higher label  
       than the current login.  
 *Action:  Ask the database administrator to perform the operation or grant  
       the required privileges.  
       For Trusted Oracle users getting this error although granted the  
       the appropriate privilege at a higher label, ask the database  
       administrator to regrant the privilege at the appropriate label.  
 > SELECT *  
 FROM  app1.t1  
 ORDER BY artist_id  
  ARTIST_ID ARTIST_NAME                     
 ---------- --------------------------------------------------  
      1 Pink Floyd                       
      2 The Smiths                       
      3 The Beatles                      
      4 Foo Fighters              
This produces the expected "insufficient privileges" error.

Now run the package as firefight:
 > CONNECT firefight/firefight  
 Connected  
 > exec app1.t1_api.do_insert('Metallica')  
 anonymous block completed  
 > SELECT *  
 FROM  app1.t1  
 ORDER BY artist_id  
  ARTIST_ID ARTIST_NAME                     
 ---------- --------------------------------------------------  
      1 Pink Floyd                       
      2 The Smiths                       
      3 The Beatles                      
      4 Foo Fighters                      
      5 Metallica         
The insert was performed as expected.

Although a synonym is the easiest way to ensure invokers rights code works as expected, sometimes environment configuration means that it is not possible.
As demonstrated above, SQL scripts to create the objects can be prefix the name of the object with the schema the update should be applied to, allowing the read_only user to be granted execute privileges on the package.

Wednesday, November 28, 2012

Table function

I have been thinking about using table functions to parse strings and use the results in joins to other tables. Table functions can be used to make PL/SQL collections behave like tables.

How efficient are they and what effect can they have on a plan?

I am using 11.2.0.3 for these tests

The function is created as follows:
CREATE OR REPLACE TYPE t_string_table IS TABLE OF VARCHAR2(32000);
/
show errors

 CREATE OR REPLACE FUNCTION string_token(pi_input IN VARCHAR2,  
                     pi_del  IN VARCHAR2 DEFAULT ',')  
 RETURN t_string_table   
 PIPELINED  
 IS  
   l_idx  PLS_INTEGER;  
   l_list VARCHAR2(32767) := pi_input;  
   l_value VARCHAR2(32767);  
 BEGIN   
   LOOP  
     l_idx := INSTR(l_list,pi_del);  
     IF l_idx > 0 THEN  
       pipe ROW(LTRIM(RTRIM(SUBSTR(l_list,1,l_idx-1))));  
       l_list := SUBSTR(l_list,l_idx+LENGTH(pi_del));  
     ELSE  
       pipe ROW(LTRIM(RTRIM(l_list)));  
       EXIT;  
     END IF;  
   END LOOP;  
   RETURN;  
 END string_token;
 /
 show errors

Now we are good to go. First thing to establish is what cardinality is estimated by Oracle when calling the pipelined function: I am passing in 5 values
 var str  VARCHAR2(4000);  
 var delim VARCHAR2(1);  
 exec :str := '10,11,12,13,14';  
 exec :delim := ',';  
 SELECT /*+ gather_plan_statistics monitor bind_aware */
 FROM  TABLE(string_token(:str,:delim));  

Running the trace through TRCA outputs the following plan (edited to remove information about the reads):
 ID  PID  Card   Rows    Row Source Operation 
 --- ---- ------ ------- ----------------------------------------------- 
  1:  0   8168   5       COLLECTION ITERATOR PICKLER FETCH STRING_TOKEN
The estimated cardinality (8168) almost matches the block size which I am using in my database (8192).

What about a more complex example?

I have created two tables which can be used for this test

 exec dbms_random.seed(0);  
 create table from_table  
 pctfree 99  
 pctused 1  
 as  
 with generator as (  
   select --+ materialize  
       rownum id  
   from  all_objects  
   where  rownum <= 100  
 )  
 select  
   rownum       id,  
   CASE WHEN MOD(rownum,10) < 5 THEN 4 ELSE MOD(rownum,10) END num,  
   lpad(rownum,10,'0') small_vc,  
   trunc(100 * dbms_random.normal) val,  
   rpad('x',100)    padding  
 from  
   generator  v1,  
   generator  v2  
 where  
   rownum <= 1000;

 CREATE UNIQUE INDEX i_from_table_01 ON from_table(id);  
 -- Gather stats 

create table join_table
pctfree 99
pctused 1
as
with generator as (
    select  --+ materialize
            rownum  id
    from    all_objects
    where   rownum <= 1000
)
select
    rownum              id,
    CASE WHEN MOD(rownum,10) < 5 THEN 4 ELSE MOD(rownum,10) END num,
    lpad(rownum,10,'0') small_vc,
    trunc(100 * dbms_random.normal) val,
    rpad('x',100)       padding,
    (MOD(rownum, 1000) + 1) ft_id
from
    generator   v1,
    generator   v2
where
    rownum <= 10000
;

CREATE UNIQUE INDEX i_join_table_01 ON join_table(id);
CREATE INDEX i_join_table_02 ON join_table(ft_id);
-- Gather stats

Now the table function is used to create a result set which joins to the from_table.
exec :str := '10,11,12,13,14';  
exec :delim := ',';  
SELECT /*+ gather_plan_statistics monitor bind_aware */
 FROM  from_table ft  
 JOIN  TABLE(string_token(:str,:delim)) ON column_value = ft.id  
 JOIN  join_table jt ON ft.id = jt.ft_id;

ID   PID   Card    Rows               Row Source Operation              
--- ---- ------ ------- ------------------------------------------------
 1:    0  81680      50 HASH JOIN
 2:    1   8168       5  COLLECTION ITERATOR PICKLER FETCH STRING_TOKEN
 3:    1  10000   10000  HASH JOIN
 4:    3   1000    1000 . TABLE ACCESS FULL FROM_TABLE
 5:    3  10000   10000 . TABLE ACCESS FULL JOIN_TABLE

In this case, Oracle assumes that the best way to get the data from the tables is by two full table scans - note that the discrepancy between the estimated Card (8168) and the actual returned (5).

Now add a cardinality hint saying that a handful of rows are returned. Note that I have rewritten the query to use an inline view.


 exec :str := '10,11,12,13,14';  
 exec :delim := ',';  
 SELECT /*+ gather_plan_statistics monitor bind_aware */
 FROM  from_table ft  
 JOIN  (SELECT /*+ cardinality(t 5) */* FROM TABLE(string_token(:str,:delim))t) ON column_value = ft.id  
 JOIN  join_table jt ON ft.id = jt.ft_id;  

ID   PID   Card    Rows               Row Source Operation               
--- ---- ------ ------- -------------------------------------------------
 1:    0     50      50 NESTED LOOPS
 2:    1      5       5  NESTED LOOPS
 3:    2      5       5 . COLLECTION ITERATOR PICKLER FETCH STRING_TOKEN
 4:    2      1       5 . TABLE ACCESS BY INDEX ROWID FROM_TABLE
 5:    4      1       5 .. INDEX UNIQUE SCAN I_FROM_TABLE_01
 6:    1     10      50  TABLE ACCESS BY INDEX ROWID JOIN_TABLE
 7:    6     10      50 . INDEX RANGE SCAN I_JOIN_TABLE_02

A far better plan.

To summarize, consider using the CARDINALITY hint to tell Oracle how many rows are expected to be returned when using pipelined functions where possible. It should lead to better plans being used for your queries.

Monday, November 19, 2012

Does an index work without a where clause? (part 1)

I was looking at some of the queries that the front end runs against one of the databases I support. The query currently does a full scan of one of the fact tables in order to generate a small set of static information - performance is acceptable at the moment but will gradually slow down as the fact table gets bigger.

Unfortunately amending the data model is not an option at the moment so I started thinking about other ways to do speed up the table. Could I use an index?

Tests are carried out on 11.2.0.3.

What is the performance like with a small table?

 CREATE TABLE small_tab
 AS
 SELECT *
 FROM  all_objects
 where rownum <= 70000;
 -- Gather statistics
SELECT DISTINCT object_type, object_type||' text'  
 FROM  small_tab;  
--------------------------------------------------------------------------------
 | Id | Operation         | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
 --------------------------------------------------------------------------------
 |  0 | SELECT STATEMENT  |           |       |       |  307 (100) |          |
 |  1 | HASH UNIQUE       |           |  39   |  351  |  307  (2)  | 00:00:04 |
 |  2 |  TABLE ACCESS FULL| SMALL_TAB | 76220 |  669K |  304  (1)  | 00:00:04 |
 --------------------------------------------------------------------------------

What if I add a normal index?

 CREATE INDEX i_small_tab ON small_tab(object_type, object_type||' text');
 -- gather statistics
 SELECT DISTINCT object_type, object_type||' text'
 FROM  small_tab;
 --------------------------------------------------------------------------------
 | Id | Operation         | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
 --------------------------------------------------------------------------------
 |  0 | SELECT STATEMENT  |           |       |       |  307 (100) |          |
 |  1 | HASH UNIQUE       |           |  39   |  897  |  307  (2)  | 00:00:04 |
 |  2 |  TABLE ACCESS FULL| SMALL_TAB | 76220 | 1711K |  304  (1)  | 00:00:04 |
 --------------------------------------------------------------------------------

Still a full scan of the table. Note that I am also unable to hint Oracle to use the index. I might take a closer look at this in a further post.

What about a bitmap index? Logically this should provide the best performance because of the way a bitmap operates internally. But is this the case?

 CREATE BITMAP INDEX i_small_tab_bmp ON small_tab(object_type, object_type||' text');  
 -- Gather statistics  
 SELECT DISTINCT object_type, object_type||' text'  
 FROM  small_tab;
 ------------------------------------------------------------------------------------------------
 | Id | Operation                   | Name            | Rows  | Bytes | Cost (%CPU)| Time     |
 ------------------------------------------------------------------------------------------------
 |  0 | SELECT STATEMENT            |                 |       |       |   7 (100)  |          |
 |  1 | HASH UNIQUE                 |                 |  39   |  897  |   7 (29)   | 00:00:01 |
 |  2 |  BITMAP INDEX FAST FULL SCAN| I_SMALL_TAB_BMP | 76220 | 1711K |   5  (0)   | 00:00:01 |
 ------------------------------------------------------------------------------------------------

... and Oracle uses the bitmap index.  Response from the bitmap on my system is about 3 times as fast as well - approximately 0.01s rather than 0.03s when using a full scan.

My next post will summarize the same tests against a large table of approximately 2,000,000 rows.