The SAP Lock Concept

This SAP article will guide you all about “The SAP Lock Concept” with screenshots.

This article will cover the following topics

1. What is SAP lock and how is SAP lock different from database lock?


2. SAP lock/enqueue solution architecture overview


3. SAP lock operation health monitoring
   
   1. Monitoring various SAP lock fill levels
   
   
   2. Monitoring SAP lock rejected rate
   
   
   3. Current top users which is holding most of SAP locks
   
   
   4. Monitor old/stuck SM12 lock entry
   
   


4. SAP lock issue trouble-shooting
   
   1. SAP lock issue overview
   
   
   2. Delete Stuck/orphan SAPSM12 lock entries
   
   
   3. Application program lock collision – causes and fixes
   
   
   4. SAP Lock table overflow – causes and fixes
   
   

1 What is SAP lock and how is SAP lock different from database lock?

SAP lock is an application protocol to ensure that one business object is modified/changed by one user or one execution of one transaction or one job at one time. Application would place request to lock an object like sales order, the application can proceed with the change logic only lock request is granted, after changes is completed, the lock on the business object would be released. A granted SAP lock request is held in an in-memory central lock table with a configurable size via SAP transaction RZ10. A lock request can only be granted when there is no existing lock related to the business object in the central lock table.

SAP lock is different from database lock. The SAP lock is on business object level. One business object can be stored in many tables. SAP lock is an application protocols – which means an application can still change a business object which is locked by another application if its’ design is to ignore the protocols. So it is kind of “soft lock”. Database lock is at entry or table level which is a “hard lock”. No SAP application can change a table record if there is a database lock on it.

2 SAP Lock/Enqueue solution architecture overview

Understanding the SAP lock solution architecture can help us to do performance trouble shooting performance issue related to SAP lock.

A typical SAP system consists of one or more application instances/servers and Central Instance. In this case, SAP lock communication path is: the lock request from the program is sent to local dispatcher which passes it to Message server which is on the CI. Then message server engages the dispatcher of CI, which forwards the request to the Enqueue work process where SAP lock request is executed.

Figure 1 SAP enqueue/lock solution architecture

Please take note that I have not listed backup file used to keep locking entries owned by updating process. Also, Figure 1 is based on a classical SAP system which consists of a central instance and several dialog instances. SAP work processes on central instance which have direct access to lock table would bypass dispatcher, message server and lock server.

3 SAP Lock Operation Health Monitoring

SAP Lock operation – health monitoring is to make sure that there is no risk of lock table overflow and no lock entry remains in table beyond reasonable duration. You monitor those areas via SM12 built-in features – Enqueue statistics and “Top capacity usage”.

Please refer to below figure 2 for explanation of SM12 Enqueue statistics. You can click here to know how to run SAP SM12 in full details.

Figure 2 SAP SM12 Lock Statistics

3.1 Monitor various SM12 Fill levels

We need to make sure that there is enough “Idle” lock capacity defined by the gap between Maximum number of “Lock Entries”/”Lock Arguments” and Maximum Fill level, so there is no risk of lock table overflow. I would prefer a maximum 90% utilization as a threshold for further investigation. We also need to pay attention to current fill-level. If current fill-level is around maximum fill level, then it might be a time to closely monitor the lock operation. Maximum Fill level is only meaningful after system has been up and business operation has gone through a full cycle.

3.2 Monitor SAP lock Rejected rate

Rejected rate is calculated as Rejected/”Enqueue operations” X 100%. You need to be aware of normal accepted rejected rate of your system. In my system, normal rejected rate is around 1%. If rejected rate goes significant higher, this can indicate a system wide SAP lock issue. The number of enqueue operations and the number of rejected is the accumulation data since latest start of the system. To see current rejected rate – you can take two snapshots of SAP lock statistics at two different moments to calculate the rejected rate for the period.

Following chart is an example I used to verified whether SAP lock rejected rate is normal. There was an application enqueue issue which impacts several business function areas for several days. The issue was resolved but someone was questioning why rejected rate was still high. I explained that data showed in Rejected field was an accumulated data – then I produced a rejected rate for a latest period to show “current” rejected rate which was at 1.5% – that calmed concern whether we were still having lock issue.

3.3 Monitor current Top SAP lock capacity user

Any single user/program should not generate more than 5% of maximum number of “lock entries”/”lock arguments” normally. We need to check who are placing most of locks when fill level is close to quotas like Maximum number of lock entries allowed. Then we can review whether number of lock entries can be reduced via changing program code or the way which the program/job is executed. There is no one-size-fits-all approach to decide what is acceptable number of locks which can be placed by one program…The bottom-line is that program logic has been designed properly in terms of SAP lock operations – lock as late as possible and release as soon as possible to avoid lock table overflow.

3.4 Monitor old/stuck SM12 lock entry

This is to make sure that old entries should be removed promptly. This has not been significant in my experience from system/application performance point view. But it could be important f number of entries stuck are significant or old lock entry is related to a frequent access object.

4 SAP lock issue trouble-shooting

SAP lock issue can cause SAP application program/job/process termination due to failure of securing a lock. Or it can be slow down a business process due to a lot of time spent in getting lock. It can cause impact all users and business operations due to shortage of SAP work process when many SAP work processes are stuck in SAPLSENA program.

4.1 SAP lock issue overview

Based on SAP lock architecture showed in Figure 1, we can know theoretically lock performance issue can be due to ill function of following components:

1. Program


2. Dispatcher


3. Message server


4. Lock server


5. Lock table


6. Application server


7. Network

You can use SM12 diagnosis function to validate whether a SAP lock problem is due to dispatcher, message server, lock server, application server or network:

1. SAP SM12 -> Goto/Extra -> Diagnosis/Diagnosis in Update and


2. SAP SM12 -> “test” in OK/command window -> Error Handling -> Diagnosis Environment.

You can use SAP transaction SM21 to review system log for SAP lock issue or use ST22 to review core-dump of a program due to lock failure.

In reality, most of performance issues related to SAP locking operation are due to application logic, slowness of SAP lock release, stuck/old SAP lock entries and lock table overflow.

4.2 Delete stuck/old SAP SM12 lock entries

It often happens that SAP lock entries can be left in the lock table due to malfunction of SAP system operations. So what is a big deal if this happens? How critical and how urgent should we deal with such stuck SAP lock entries? This really depends on particular situation:

• Volume of stuck entries :– If a huge number of lock entries are stuck can lead to SAP lock able overflow. If a huge number of lock entries from many different users, this can indicates Malfunction of SAP system.

• Lock attribute of stuck entries – share lock, exclusive lock, lock at table level and lock at single entry level has totally different meaning to follow SAP lock operation on the object. For example, lock on the entry level only impact subsequent SAP lock request which need to lock the same entry. Lock at table level would impact any subsequent SAP lock request which requests a lock on any entry of that table.


• Business nature which lock object represents – a lock on infrequent access object and a lock on a frequent access object would have different impact to normal business operation which needs the lock.


• Nature of business operation – a lock entry for a single business document related to archiving process is left in system. Since normal business application has no need to change the business document again, the impact to business operation in the system is “zero”. However if a lock entry is related to open customer rush order which is subject to change in subsequent business process, then a stuck entry would stop subsequent activities which needs to update the order. In case, this is an urgent/rush customer order, then it is time critical to address the stuck entry.

A stuck SAP SM12 entry can cause subsequent SAP lock request failure if the subsequent lock request needs to lock the same object. So it should be removed to avoid business process fallout/delay. But how do we know a SAP lock entry can be safely deleted? How old is old enough? Following is my tips to safely delete a stuck SAP SM12 lock entry:

• SM12 Lock entry is not owned by updating task( backup flag is not checked)
  
  • Orphan SAP lock entry can be deleted – Orphan means owner of this lock is not active in the SAP system. You can find owner information from SAP lock entry detail – especially, SAP host name, user name, SAP work process#. SAP work process# has a process type (background, Dialog etc.).
    
    • If the lock is issued from a background work process on a server but that “BGD” work process is handling request from different user now or that “BGD” work process is in idle status, then the lock is an “orphan” lock and can be deleted. So on.
    
    
    • If the lock is issued from a dialog work process on a server and the related SAP user is not logon in SAP system( SAP SM04 and AL08 to check logon users). Then the lock is an orphan entry. Please pay attention that a SAP “DIA” work process is designed to be shared among online users which might execute different SAP transactions/programs – an user task would be rolled out of a SAP dialog work process during the online user’s “think”/”pause” time. For that reason, it is normal that an online user have locked a sales order, but you do not see any work processing running under that user in SAP SM50 and the process from which the lock is issued is in “waiting” status. So you cannot say since the work process is idle now, then the sap lock entry is “orphan”. But you properly can use “date time ” of lock entry details to say whether a sap lock entry is orphan if your system has a timeout setting for inactive online user and process.
    
    
  
  


• SM12 Lock entry is owned by updating task( backup flag is checked or shadow/blue entry)

In General, you do not delete an SM12 lock entry whose backup flag is checked. The SAP lock entry will be removed automatically after related updating task is completed. However in following situation, a lock entry can be deleted safely.

• Any SAP SM12 lock entry older than updating task retention period can be removed


• A SAP SM12 lock entry can be deleted if no outstanding updating task exists in SAP updating task monitor SM13. For example, SAP SM12 lock entry is under user A, but SAP SM13 update backlog contains no entry from user A. In this case, this SM12 entry can be deleted.

Improper deleting SM12 lock entry might cause data integrity issue for related table or business object. Make sure that you have checked SAP system, job and user to reach a conclusion that it is safe to delete a SAP lock entry.

4.3 Do trouble-shooting on SAP lock/enqueue collision – SAPLSENA

4.3.1 THE SAPLSENA ISSUE DESCRIPTION

SAP lock/Enqueue issue related to program SAPLSENA is the most often-seen and tricky SAP locking issue. When this issue happens, one or many Sap work processes are continuously running with report SAPLSENA in SAP work process monitor SM50 screen( refer to Figure 3 to see sample screen) and SM66 screen( refer to Figure 4 to see sample screen ). In worst case, all SAP work processes are occupied by report SALSENA program for a period. Those SAPLSENA work processes are keeping requesting to get a SAP lock. So in nature, this issue would impact program/process which needs to change business object when SAP system still have enough “idle” work processes.

This issue itself is an application issue since SAP lock is an application protocols implemented in business application/program. In some cases, the issue can disappear by itself without any human interaction if you wait “enough” time. It could be normal that some of SAP work processes are running with program SAPLSENA for a brief moment. It becomes an issue when a SAP process is seeing running with SAPLSENA for a significant time and it has a material impact on runtime of a business transaction.

Figure 3 SM50 – many processes are showing SAPLSENA under report

Figure 4 SM66 – Many work process with a ENQ status and SAPLSENA program

4.3.2 WHY DOES THIS ISSUE HAPPEN?

In online transaction, if you are going to use SAP transaction va02 to modify a customer order which someone else is changing it via the same transaction or other standard SAP transaction, your transaction would be aborted by SAP and telling you that another user else is changing it. Then you can only change the order again after the user’s change on the order is completed. In situation like this, what you have seen in figure 3 or figure 4 would never happen since lock request duration is under a few milliseconds, you normally do not see program SAPLSENA displayed in SM50/SM66 screen.

In background processing, SAP application can have a logic of keeping trying until lock is secured assuming whoever holds the lock would release the lock quickly. This type of design is to avoid interruption of background process execution.

If the program who holds the lock cannot release the lock quickly for some reasons or there are huge number of requests to lock the same object, so the lock request keeps retrying, that makes program SAPLSENA visible in SM50/SM66 like figure 3 or figure 4 would be observed in SM50/SM66 and many business transactions/processes would be put on hold due to this.

Figure 5 is a standard SAP code, Program would execute the DO-ENDDO loop until the lock request is successfully placed.

Figure 5 DO-ENDDO loop for lock requests

Please note that I am explaining the technical reason why lock requestor is seen

4.3.3 HOW TO DO TROUBLE-SHOOTING WHEN SUCH ISSUE HAPPENED?

When this issue happens, it is important to understand what business object is involved and who is holding the lock. To find out such information, you have two ways

Log enqueue operation via SAP transaction SM12 – you can find out business object (lock object) involved in collision based on SM12 log. Please remember to switch off logging after you are done to avoid disc space issue. Several minutes log should be enough. Please refer to SAP Note 125041 to know SAP enqueue log format.

Do an enqueue trace via SAP transaction

ST05/ST12 – This is my normal way to verify what is collision and who is holding the lock which block the process being traced. You can base on collision owner to find out server/instance and work process related to lock holder process. Collision object is a lock object – you can use SAP SE11 transaction to check lock object “description” to find related application area. You can contact the SAP user to find out more about the application related to the lock holder. Based on those information, you might identify the offender, review possible solution or next step. Please refer to Figure 6. Figure 6 is an enqueue trace related to one SAPLSENA performance issue where all SAP transactions which related to order creation and order changes were stuck. Based on the trace, I was able to identify and later confirmed that SAP CIF model activation job was the root cause.

Figure 6 – Enqueue trace statement details

In our production box, our users were complaining slowness of deleting workflow item via SAP transaction SBWP. In SM50, many processes are stuck with program SAPLSENA. This was caused by an application setting which was causing lock contention. You can refer to my post “SAP SBWP workflow item deletion performance issue“.

4.4 SAP lock table overflow

4.4.1 WHAT IS SAP LOCK TABLE OVERFLOW?

In what is a SAP lock section, it is stated that a granted SAP lock is held in an in-memory table whose size is specified via SAP parameter enque/table_size. So there is a limit on how many lock requests can be hold in the lock table. When that limit is reached and a program is making a new lock request, the SAP system would send “lock table overflow” error message to the program.

SAP lock table overflow is really a message that SAP lock table has been full and cannot accept new lock request anymore.

4.4.2 WHAT ARE POSSIBLE REASONS FOR LOCK TABLE OVERFLOW AND HOW TO FIX SAP LOCK TABLE OVERFLOW ISSUE?

When SAP lock table is overflow, then no lock can be granted since no empty slot in the lock table to hold the lock entry. All programs/jobs who need to place the table would fail. Apparently this happens rarely to a well-tune SAP system.

Lock table overflow can happen due to one or more of following reasons:

Small SAP Lock table is too small – due to increasing of business volume and activities, the lock table size might need to increase. SAP lock table size is controlled by SAP parameter enque/table_size.

Slow release of SAP lock – SAP lock entries are not released timely due to system/application performance issue. For example, if SAP updating function is slow or stopped, all entries which belong to updating tasks would remain in SAP lock table until corresponding updating tasks are completed. In this case, increasing lock table size is just addressing the symptom not root cause.

Improper application design on SAP lock – Application design of top SAP lock capacity user should be carefully reviewed especially if it is using high % of SAP lock capacity like 20%. When I worked on an archive project, a program used over 50% of SAP lock capacity due to faulty design and caused overflow. Please refer to my post – SAP job cancelled on error “The document cannot be blocked” to know more on this case.

Improper job/application schedule/deployment – If two jobs which would place a lot of locks and run at the same time can create lock table overflow, then these two jobs should run separate time windows if business logic allows.

Inappropriate execution – This is referring to the situation where a user is executing a transaction with abnormal volume due to accidental operation or process a huge volume in a time-window which is not allowed.

Program performance – If a program is not coded/design properly, it could hold a lock unnecessary longer. When the program is holding a lock on a common accessed business object, this would create lock issue. Solution in this case is to tune the program performance.
4.4.3 HOW TO KNOW WHICH ONE IS TRUE CAUSE?

You need to use information provided by SAP SM12 SAP top capacity used – history screen which captures top offender when overflow happens. And start to do investigation with that. Please refer to Figure 7 for the history screen.

Figure 7 SAP SM12 Top capacity used – History

Being familiar with your system and the normal fill-level and high-water mark for various fill levels, this would help you to do trouble-shooting as well. If NR_of_locks held by top offender is reasonable, then your lock table size might be too small. You need to increase it. If your system is really slow or overloaded when the lock table overflow happens, then you have to focus on system performance issue first – in my experience, this has not been a case so far.

Further clarification

Typical SAP lock performance issue described in this post is based on my performance experience. Situation in your system might be difference. I would like to highlight again that you should take caution to delete an SAP lock entry and switch off SAP enqueue logging. My personal preference is to use SAP ST12/ST05 enqueue trace, I found myself often lost in the huge number of enqueue log entries and cannot find lock holder.

When SAP updating is slow for whatever reason, an end business user cannot change a document which they have changed “not long ago”, you cannot delete a SAP lock entry to make business “happy” – this create data integrity issue. Right direction is to understand why updating is delayed and wait for completion of updating.

When you have a “system wide” lock issue with many SAP work processes running with report SAPLSENA and your SAP system is not overloaded, this “system wide” lock issue is caused by application/program/job instead of system.

Now, you must have got clear idea about “The SAP Lock Concept”.

 

The SAP Lock Concept

	12:26:00(EST)	10:52AM(EST)	Period defined by two time
Enqueue operations	698,445,905	697,164,953	1,280,952
Rejected	255,946,432	255,927,063	19,369
Rejected rate	36.6%	36.7%	1.5%