Recently I was involved in reducing SAP SM21 system log. One of top messages in our SAP SM21 log is “Terminal ##### in status DISC” & “Delete session ### after error ###”. About 20 such messages were generated hourly under a particular user. I looked into this and fixed the issue. I also looked into other SM21 log messages – “canceled transaction” and perform “rollback”. In this blog I would share my understanding on those errors and how those errors are addressed.
- SAP SM21 log – “Terminal ##### in status DISC” & “Delete session ### after error ###” ,
- SAP SM21 log – Perform Rollback ” “Canceled transaction” and
- SAP SM21 log – “Canceled transaction”.
1 SAP SM21 log – “Terminal ##### in status DISC” & “Delete session ### after error ###”
1.1 Overview of investigating “Terminal in status DISC” & “Delete session after error ###”
“Terminal in status Disc” means a broken connection between SAP application servers and the frond end like SAP GUI etc. “Delete session after error ####” in this case, the error is 023. The error means execution of database operation is terminated before it can complete normally. In our case, HP quality center is used to monitor system status. VuGen scripts are executed from several servers to logon SAP to simulate online user transactions. The solution is to make sure that script is executed sequentially instead of parallel and let the script to wait for completion of SAP transaction before it exits from SAP. And that solution has fixed the issue. Details are covered in following sections
1.2 The SM21 log
Following is a combination of SM21 SAP system log screens and STAD transaction statistics screen for the same period – 01:00 – 02:00. Both screens are truncated. The SAP STAD is clearly showing that SAP transaction VA03 was repeatedly executed. 1st “Terminal 00397 in status DISC” was logged at 01:07:34. Prior to that, there were several VA03 executions without SM21 log message.
Figure 1 SAP SM21 Terminal in status DISC
Further reviews in other hours indicated that this SM21 log happened hourly but within an hour, it occurred randomly and logs are related to 5 different terminals.
“Terminal in status DISC” and “Delete session 001 after error 04” is one pair of message. “Deletion session 001 after error 23” comes alone. Please refer to figure 2 for sample.
Figure 2 SAP SM21 Delete session after error 023
1.3 What do SM21 log -Terminal in status DISC and Delete session 001 after error 023 mean?
Terminal in status DISC means that the SAP system is trying to send data to the terminal(client) but the terminal is DISConnected from SAP. This could be many reasons. Analysis of statistics records (SAP STAD) under user HPMONITOR indicated that va03 transaction was executed from several clients/terminals. Owner of this execution confirmed those va03 transactions are executed via VuGen(Virtual User Generator) scripts of LoadRunner. Analysis of statistics records indicated there was concurrent execution of those VuGen scripts. All execution of VuGen Scripts are using the same SAP dialog user account. All those scripts logon SAP under the same dialog user. SAP user license agreement does not allow sharing user account – or the same userID logons to SAP system twice at the same time. I think that concurrent logon via VuGen Script from different PC could contribute to our issue. Based on that, we re-arranged the execution of Vugen scripts to ensure that only one script at one time from one location is executed.
With above changes, the Terminal in status DISC log did not happen hourly any more, but it did happen occasionally on a daily basis. We had another error message “Delete session 001 after error 023” which was then number 1 message logged under the user HPMONITOR. SAP message on error 23 said “process restarted”.. really does not give me any direction, so I reviewed the developer trace. Following is the screen shot of Error message log and developer trace (access it via SAP transactionSM50).
Figure 3 Delete session after error 023 and SQL error 1013
From developer trace, we could see that oracle error 1013 is logged with the “Delete session 001 after error 023”. SQL error 1013 happens due to “user requested cancel”. ORA-01013 is “user requested cancel of current operation, This forces the current operation to end”. There is no cancellation command in the script and message was logged randomly with the execution of the same script. Why would execution of the script have such behavior?
I think this might be due to the fact that the script was sending a logoff signal to SAP system from the script while operation in SAP had not completed, Based on system status, the same operation in SAP could be faster or slower. The logoff signal from the script can arrive SAP faster and slower depends on network traffic.. so it looks like to me – this happened when logoff signal from the script arrived SAP prior to completion of previous step. It should help to fix this to add some wait in the Vugen Script prior to logoff. So the VA03 Vugen Script was modified by adding a few seconds wait prior to step of logoff.
With above changes, the execution of those scripts generates no SM21 log any more in our system. As what is showed in following screen shot of combining of SM21 log( no logs under HPMONITOR) with execution of those scripts(STAD).
Figure 4 SM21 – Terminal in DISC – issue fixed.
1.5 How to reproduce SM21 message log – “delete session after error 023”
Logon SAP via SAP GUI from your Microsoft PC, Run se16 against a big table querying data without using key/index field, while it was still querying data, terminate your SAP GUI via window task manager, then you would reproduce the same SM21 log and developer trace showing SQL error 1013 like what is showed in Figure 5.
Figure 5 SM21 – simulate SQL 1013 error
Alternatively, I believe I can simulate the message “Deletion session ### after error 023” by terminating a running SAP work process in middle of SQL operation etc.
There are similar SM21 messages like “Connection to user, terminal lost”. The root cause is normally outside of a SAP system but related to specific user/program behaviors.
2 SM21 log – Perform Rollback
Here I would show how to trace back to the job/program which is producing SM21 log – “Perform rollback” or how you can find parent of SM21 message “Perform rollback” via an example.
Steps used to find the SM21 log’s parent — SM21 -> /sdf/mon -> SM37 -> WE02/WE05
You can use information timing, User, work process number , server to search job/program name in performance snap-shots generated via SAP transaction /SDF/MON…as showed in Figure 6.
Figure 6 SM21 log and /SDF/MON – Job which generated “Perform rollback” log
Then you can use SM37 to find specific job as showed in Figure 7.
Figure 7 SM37 Job details
Then you can check job log or job spool depends on situation from SM37. Here is the job spool
Figure 8 SAP job spool
Figure 9 Job spool details
From figure 9, you can find the idoc 0000001390670257 is in “51” status and not posted. Using WE02/05 or se16, you can check the IDOC status message, you can see that IDOC was processed again and again, each time it was issuing “perform rollback”. Based on IDOC, you can find corresponding function module which is used to load the IDOC..
Figure 10 Idoc status – technical info
Figure 11 Idoc was repeatedly processed
3 SM21 – transaction canceled
Transaction can be canceled due to many reasons like program code (like “A” type message instead of “E” type), SQL error (like DBIF_RSQL_SQL_ERROR etc.), memory issue ( like System_No-Roll etc ), manual cancellation and so on.. Following are screens showing “transaction Canceled” message.
Figure 12 SM21 – transaction canceled
If you would like to know how to trace it back to know what job and program is related to a particular line of “Transaction Canceled” and there is no core-dump. Here are the steps:
SM21 (Timing, user, SAP work process, server) -> /SDF/MON ( timing, server job name, work process) -> SM37 ( verify job status )
Figure 13 SM21- Transaction canceled and its’ parent
When there is a core-dump related to SM21 log, you can use SAP transaction ST22 to find the parent of a SM21 log. With job name, timing, server name and work process number, it is possible to associate a SM12 log message to a specific job execution via SAP transaction SM37. Figure 14 is the specific job instances related to highlighted entry in Figure 13.
Figure 14 SAP SM37 JOB details
“No deliveriy items found” message showed up in the job log (Figure 15) and message type is “A”. So that is why the message went to SM21 log. So this is an application issue.
Figure 15 SM37 job cancelled under SAP message type “A”
Figure 16 shows that whenever the job was canceled, a related message was logged by the SAP system.
Figure 16 SM21 log and SM37 Job
From Figure 16, we knew that the job was executed every 20 minutes and it almost failed every time. Apparently, this was an application issue. In further analysis, I found that this was due to the same problematic IDOC. To get the message disappear from SM21 log, it is a simple fix but that would not solve the data issue as well as the program design how to solve exceptional case. From job setting point view, the frequent job cancellation can be avoided by excluding problematic IDOC from being processed each time.
In this post, I shared several cases on how to review SM21 message log and trace it back to its’ originator and solution to fix this. Sometimes, the SM21 message are logged even everything went Normal in end business view( in our case HPMONITOR ), but most of cases, users should notice something abnormal or can be monitored from application point view like SAP SM37 job cancellation. Monitoring job cancellation via SM37 and fixing job cancellation could make corresponding SM21 log disappear if there is any. So I am not suggesting that we should start with SM21 to fix an issue. The preferable starting point is that application should monitor health of their operation closely and fix any issue properly… Even there is SM12 log related to an operation, but it could be all right or nothing worth to be worry about normally.