How to load LOB tablespaces using DSN1COPY?

In the past, I've tried to load tablespaces with LOB columns using the regular IBM Load, BMC Load Plus & CA Fast Load. None of these tools support LOB tablespaces. You get into a series of errors.

The only utility that has ever worked for me is DSN1COPY. It's a pain to use DSN1COPY though. You need several things to go right for this to work. Let me try to explain as simple as possible.

In this example, I'm copying data from an imagecopy dataset into a table in a TEST subsystem. The first thing you need to remember is that, you need to know the DBID of the database, PSID of the tablespace and OBID of the table for both the source and target subsystems.

These are the steps:

1. STOP the base and LOB tablespaces and indexes for the LOB table on the target subsystem

2. Run DSN1COPY with the right parameters. I use PARM='PAGESIZE(32K),LOB,FULLCOPY,OBIDXLAT,RESET'. Supply the imagecopy dataset of the table on the source subsystem for SYSUT1 and supply the DSNDBC linear dataset of the table on the target subsystem for SYSUT2.

3. START the base and LOB tablespaces and indexes for the LOB table on the target subsystem

4. Rebuild the indexes

5. Run runstats

If the DSN1COPY step abends with "DSN1992I VSAM PUT ERROR, RPLERREG = 008, RPLERRCD = 028", check JESMSGLG. If you see a message like this "IEC070I 204-220,jobname,stepname,SYSUT2,6BF7,DBD115& IEC070I DB2CATx.DSNDBC.dbname.tsname.I0001.A001", it means you do not have enough space in that dataset. Make sure, you assign multiple datasets with suffix A001, A002, etc. in the target subsystem, it needs to accommodate all data from the source.

If you see a message like this "IEC070I 209-220,jobname,stepname,SYSUT2,89B5,DBD235,& IEC070I DB2CATx.DSNDBC.dbname.tsname.I0001.A001,", that means no space is available in the DASD volume. To fix this problem, make the dataset multi volume so that it can extend space into multiple volumes.

HTH !!

DB2 for zOS Index on Expression

DB2 for z/OS version 9 brought in a new feature called "Index on Expression". Yes, you can create indexes on frequently used expressions in your queries. For example, if you often do a SELECT FIRST_NAME WHERE UPPER(LAST_NAME) = 'SMITH', then creating an index on UPPER(LAST_NAME) will be useful.

Until V8, you can create index on just columns, not on expressions/functions etc. So, you would have created an index on LAST_NAME, but DB2 would have ignored it because when you use a function on a column, DB2 ignores index access. But starting in V9, you can create an index on UPPER(LAST_NAME) and DB2 will use this index.

Now, to the tricky part ...

I tried to create an index on UPPER(LAST_NAME), DB2 complained that I need to specify 'locale' name. After reading a little bit about 'locale names', I specified UPPER(LAST_NAME, 'En_US') and DB2 happily created the index. I also ran runstats on the table. However, when I did an explain on the query SELECT FIRST_NAME WHERE UPPER(LAST_NAME) = 'SMITH', DB2 didn't seem to pick up the newly created index. That puzzled me.

After reading a little further, I found that I need to specify the same 'locale' in the query too. After specifying the 'locale', I did an explain and DB2 picked up the index. This is the modified query:

SELECT FIRST_NAME WHERE UPPER(LAST_NAME, 'EN_US') = 'SMITH'

Keywords: DB2, z/OS, Index, Expression, Upper, Locale, function

DB2 V9 for z/OS - New ZPARM EDM_SKELETON_POOL

I'm working on upgrading DB2 for zOS from V8 to V9. I completed upgrading it to V9 CM (Compatibility/Conversion Mode) in one of our PROD subsystems about 2 weeks ago and the next business day was horrible. The performance was bad for the entire subsystem. The LPAR CPU peaked to 98% of the total MIPS capacity.

The activity rate against the SPT01 tablespace jumped several times and the CICS regions started sucking 20% more CPU. We noticed a few bad queries through Omegamon and we fixed them by either tweaking the stats or putting some new indexes. But that didn't bring down the CPU consumption much. The interesting part was, our zIIP engine started to peak at 100%, which usually stays at 40%. We found one query that had done a bunch of updates and got stuck - it was using a lot of CPU but was not doing any productive work. We cancelled this thread, but it wouldn't go away and the zIIP engine was still at 100%. Many more transactions/threads started to back log because of lack of CPU availability. The cancelled thread didn't die and still was burning CPU. We ended up cycling DB2 with a MODE(FORCE) command. Still, when DB2 came back, it started recovery of the cancelled thread, so we had to force kill it.

When that thread was at last gone, we looked at the CPU rate, and it was right back at the 98% with the only exception that zIIP processor was at 75% level.

We opened a Sev 1 PMR with IBM. As usual, they asked for dumps, SMF, RMF records etc. [Rant On] I would have thought IBM provided a solution to try for a problem with such a mature product. But instead they were waiting on us to provide docs. [Rant Off]


Anyway, we did some more research and found that, in DB2 V9, copies of packages and plans have been moved from below the 2GB bar to above the bar. This is done through a new zparm called EDM_SKELETON_POOL. The default value for this zparm is too low. So we increased it by 20 times. The system is in a much better shape now.

This is from Omegamon for DB2:
Package Table (PT) Reqs         1005483K 
PT Loads                        80073250 
% of PT Loads from DASD             7.96%


The "% of PT Loads from DASD" should be less than 10% for a healthy system. It was about 55% in our system before increasing the value of EDM_SKELETON_POOL. With any other virtual memory, make sure that there is enough physical memory to support this increase.


After this fix, the CPU looks much better but we are still seeing some runaway transactions or changed accesspath (which are not good in many cases). We're fixing them by rewriting the queries in most cases.