Kein Folientitel

1
European Patent Office Timeout Deadlock
troubleshooting Challenges and solutions
Philippe J. HERMAN pherman_at_epo.org
2

• The European Patent Office (E.P.O.)
• Set Up in 1973. (BE, DE, FR, NL, LU, CH, GB)
• Grant patents that are recognized
  internationally.
• 26 Members in 2002.
• More than 50 other countries recognizing EPO.
• 40.000.000 of Patents Documents.
• 165.000 Patent applications in 2002.
• 47.000 Patent delivered in 2002.
• 60 of worldwide patent research.

3
Romania
4
Administrative Council All Member States
President Munich

• DG1
• Research
• Examination
• The Hague
• Berlin

• DG2
• Research
• Examination
• Munich

• DG3
• Appeal
• Munich

• DG4
• Administration
• The Hague
• Munich
• Vienna

• DG5
• Legal Affairs
• Internationnal
• Munich

5

• The IT department
• 200 staff
• 500 contractors (Australia, Paris, Bangalore)
• 150 TBytes of production data
• 36 GBytes Memory
• 3000 MIPS
• 9672 - Z87 (x2)
• 9672 - R36 (x2)
• 9672 - R34

6
TCP/IP
Java

• BMC

BDAM
z/VM

• Infotel

CICS
AIX
AS400

• Candle

Tivoli
SAP/R3
Solaris

• IBM

z/OS
Vega Bridge

• Cogito

DB2
SNA
MQSeries

• TMON

IDMS
HP-UX

• Princeton

CICS Gateway
z/Linux
Notes
Windows
7
Phoenix

• Transport, storage and access of dossiers.
• Written Documents (Patent docs)
• Images
• Links
• No paper documents
• Reduce archive costs
• Tracking of documents
• Automation

8
Phoenix
IDMS DB2 z/OS
Outgoing Documents
Printing
Incoming Electronic Documents
Incoming Electronic Documents
Incoming Electronic Documents
Incoming Documents
User Interface
Email
Input Management
Scanning Indexing
Tape Management System
CR-Rom Archive
Loading Distribution
Dossiers
Messages
9
CICS
DB2
Client
LINK
Non Extended UOW 1
Task 1
Thread 1
RETURN COMMAREA
LINK
Non Extended UOW 2
Task 2
Thread 2
RETURN COMMAREA
LINK
Non Extended UOW n
Task n
Thread n
RETURN COMMAREA
10
Extended UW (the good)

• CICS External Call Interface (ECI)
• One client transaction
• Multiple CICS Host transactions (tasks)
• Multiple thread related to the same unit of work
  (one per task)
• ONE LUW (CICS TOKEN)
• Works fine
• Looks modern

11
Extended UW (the bad)

• User think time
• CICS host transactions not master
• No Syncpoint ?!?
• No Rollback ?!?
• CICS like but weak standards (open)
• Recovery done by client application
• Cross-Environment

12
Extended UW (the EPO bad)
13
Extended UW (the EPO bad)

• No serialization (Locking)
• Network drops (ZCIOWAIT)
• Authid change (IFI cut)
• Same LUW in multiple // threads (IFI mess)
• No recovery (SQL code)
• User behavior (Response time)

14
What can we do ?

• Application design improvements
• Serialize / Group
• Negative SQL Code Management
• Recovery

• Network Improvements
• Stability
• Monitoring

• User Education
• Batch Online
• The Dragoon

15
What else can we do ?

• The Crude way
• If more than 5 min of Inactive Time in DB2
• AND If ZCIOWAIT in CICS
• Then
• Cancel
• Log the cancel for statistics

16
What else can we do ?

• If a timeout happens
• Then
• Get the Lock holder
• Start the clock
• If clock gt than 5 min
• AND 1 new timeouts
• Then
• Cancel
• Log the cancel for statistics

17
What else can we do ?

• Nothing to do for the deadlocks (redesign)
• Log all the timeouts and deadlocks
• Produce a daily statistics
• Send it to the high management level (annoy)
• Associate a cost on timeouts and deadlocks
• Play politics . . .
• Asked to stop the cancel procedure (stop
  annoying)
• Let the problem grow !
• Wait for the big bang !

18
Results (so far)

• Redesign the application
• User Education
• DB2 Basics for the developers
• 4 times less timeout and deadlock so far !

19
Thank you for your Attention... ...Questions ?