In my previous post, I talked about how to do navigate through various analysis views of SAP ST03N transaction. In this post, I would give an introduction on typical types of performance analyses which I have used SAP ST03N for in my work.
- Who is placing workload into a SAP system and when/where/how is SAP workload generated?
- What is SAP system/application performance trend?
- Are there any potential resource bottle in processing workload?
- Trouble-shooting SAP performance incident.
1 SAP system workload – who, who, where and how
SAP ST03N cam tell you – who are generating workload in a SAP system, when/where/how are those loads generated in the system for the selected period – last minutes, current period (current day/week/month) and past period( a day, a week or a month). Where here means which SAP instance when a SAP system has multiple instances. Using those information, you can do a lot of performance analysis.
1.1 SAP workload distribution among different SAP tasks
In a SAP system, there are different tasks – Dialog, Background, RFC, ALE, Update task etc. If you are wondering how significant each task is in terms of CPU utilization? ST03N can answer such question. You can pull data from SAP ST03N workload Overview and visualize SAP workload distribution among different tasks via a chart similar to following Figure 1. The pie chart reflects load distribution of a SAP S2C production system.
Figure 1 SAP system application server CPU utilization distribution
The above chart clearly indicates that who (background process) accounts for 60% of CPU utilization in the elected period. In the similar fashion, you can get chart to see database time distribution among different tasks. So you can quantify level of resource which is consumed by each of SAP SAP tasks. This type of information is available at system level or server/instance level in the selected period.
1.2 SAP workload distribution among different SAP instances/servers
In a SAP system which you have multiple instances or servers, you can use ST03N to find out what is their share of processing SAP workload. You can pull data from ST03N Instance Comparison view and visualize SAP workload distribution among different SAP servers/instances via a chart similar to following Figure 2.
Figure 2 SAP workload distribution among SAP Servers
Figure 2 clearly shows SAP workload distribution among different SAP instances. So you can see whether there is a load balance issue overall in the elected period. It is expected that every server in a system should process similar amount of workload assuming all servers has identical capacity and configuration. Figure 2 shows Server 2 is handling significant more load than other servers. So next step of analysis is to see understand SAP tasks and their CPU consumption in server 2.
1.3 SAP workload distribution among different hours
If you are wondering how workload is distributed hourly in a 24-hours window, SAP ST03N time profile can help you. This can tell you overall when the system is most busy and when the system is not. This might be helpful if you need to find a time window to let your system to handle some unplanned workload or you would like to check whether you can move an unnecessary peak load to an off peak period to protect SAP performance.
Figure 3 SAP workload – hourly distribution
SAP ST03 or ST03N provide workload hourly distribution on three intervals – daily, weekly and monthly period. This would give you a better picture on SAP workload distribution.
Level of workload in a SAP system is related with business operation which is often dynamic and keeping changes, so it is normal that workload would vary from hour to hour. However if type of workload analysis detect that there is a clear pattern – for example, one particular server at one particular hour is always having much higher workload than other hours and other server, then it worth next step to look into this in details to see whether we easy the workload “hot” spot.
1.4 SAP workload distribution among different transactions and programs
Based on data from SAP ST03N EarlyWatch profile, you can know how system load is distributed among different sap transactions and programs and know what are most expensive transactions and programs in your system.
Figure 4 SAP workload distribution among different transactions and programs
This would be value input to help to choose which transaction we should focus on for general performance tuning. SAP ST03N can help you to find out whether a transaction/program has been ever executed in the interested period.
This type of SAP workload analysis can show you a list of top work load contributors. This should help you if you would like to identify a list of a programs/jobs for performance tuning.
1.5 SAP workload distribution among different sap users
Based on data from SAP ST03N User profile, you can visualize top expensive workload users and produce chart like Figure 5.
Figure 5 SAP workload – CPU time distribution among users
This would provide workload distribution at SAP user level regardless of individual business process and application. You need to review this type of data if you suspect that particular user might occurs more load than it should…
SAP ST03N can help you to find out whether a user is “active” in the interested period and what transactions and programs are executed by a SAP user in an expected time window.
1.6 RFC work load and load from external analysis
SAP ST03N provides detailed data to show you what programs/transactions are generating RFC load in the system, what users are generating RFC load in , what are those external systems which are placing RFC load in the system. So if RFC load is a significant portion of your system and you would like to analyze this, you can use data from SAP ST03N RFC profiles and “Load from External Systems” view to do further analysis to identify top contributors. Based on that, you can decide whether there is a need to take further action like performance trace and analysis etc.
Figure 6 SAP workload From External System
1.7 Load profile of individual SAP transaction/report
Performance tuning is normally to cut time, so it is important to know where time is spent by a transaction/program/job. SAP ST03N can show where time is spent by a transaction/program/job – it is spent on most of time on application server side – ABAP logic operation, spent on database operation etc. Please pay attention that this is an average data for the specified period you choose when you run ST03N. In the period, the transaction/program might be executed many times. So it might be different from time statistics of a specific execution.
Figure 7 SAP workload – transaction time statistics
Data similar to Figure 7 gives a general direction on where you should focus on tuning application performance. If a program spends 99% of time on CPU, you might need to focus on the ABAP logic instead of tuning SQL statement of this program. In Figure 7,1,024 ms out of 1,355 average response time is spent on GUI side. Since VA03 is a sales order display transaction, this could indicate a network issue or front-end issue if user is complaining of performance issue with VA03 which is normally performing normal.
2 SAP system performance trend – system, transaction
If you wonder how system performance/application performance evolves over time or you wonder how a SAP system is performing after a significant change to the SAP system like upgrading, migration, added resource, SAP ST03N can help you with this.
2.1 What is performance trend of a system
You can compare average response time per transaction step for main task type in several periods to see overall system performance trend.
Figure 8 SAP system performance trend after migration
Figure 8 is a true production data after an ECC system migration. In this particular case, the corresponding SAP system is moved from Itanium/HPUX to x86/Linux Server. Based on Figure 8, a performance improvement are seeing across all major SAP tasks after X86 migration. Correspondingly, all key business jobs we checked in our SAP environment are showing a significant performance improvement after migration. When SAP system performance trend analysis shows a trend of deterioration, then further review on runtime’s component like CPU time, database time etc. are needed to identify further analysis action.
2.2 What is performance trend of database system?
Average time taken for database access – like sequential read, direct read and change operation etc. can be a meaningful indicator for overall database performance. You can visualize database operation by producing chart to similar Figure 9.
Figure 9 ST03N workload – database operation performance
Figure 9 is a true example to show the SAP MDS operation’s impact on background task in our production in our X86 migration project. We were seeing slowness in many transactions and programs in MDS drill 1 period. One particular process was so badly impacted resulting in missing service level agreement due to implementation of SAP MDS migration solution. For more details, you can refer to my post on SAP MDS Migration and system performance.
2.3 What is performance trend of a transaction/program/job?
Transaction profile view of SAP ST03N shows average response time per transaction step in 3 type of interval – daily, weekly and monthly. You can review average response time per transaction step for a specific transaction over a long period. This could give you an indication of SAP transaction/program/job performance evolution – could be stable, or be running longer and longer, be varied but within a specific scope etc. If program is running longer and longer, then action might be needed to see what drives growth of runtime.
Figure 10 SAP workload – performance trend of individual transaction/program
Resource bottleneck analysis
3 Resource bottleneck analysis
3.1 SAP work process utilization analysis
Several views of SAP workload monitor ST03N contain a column named as “Average wait time per dialog step”. Following screen shows is from ST03N workload overview
You can check wait time at system and instance level for a specific day, week and year. You also can check “wait time” in a 24-hours fashion. Normally, wait time should be around a few millisecond. Wait time is to measure how long a job or transaction has to wait until the system can find a free SAP work process to run it. So consistent high wait time can indicate that no enough SAP work processes is configured in system or there is no enough CPU or application has occupied work processes too long due to application design if no issues like storage, network, database issue etc. put SAP work process into “abnormal” or “error” status. You can refer to my post on
SAP work process monitor
for more information on status of a SAP work process.
3.2 Other resource bottle-neck analysis
SAP workload monitor calculates “Average Processing Time” and “Average CPU time” per transaction step. This can be used to monitor “wait” situation – such as wait for free system resource – CPU, SAP memory, programmed wait/sleep due to data object lock, wait for establishing communication channel e, wait for completion of RFC etc.
CPU time is consumed during the processing time. So length of “Avg. CPU time” is related to length of “Avg. Proc. Time”. If a transaction is making “no RFC” call and no program coded “sleep” or “wait” statement, then “Avg. Proc. Time” should be closer to CPU time and should be less than 2 times of “CPU time”. If this relation is broken, then this indicates a shortage of CPU normally.
SAP ST06 can be used for CPU load analysis as well, you can check my post – How to use SAP transaction ST06 for SAP performance analysis for details.
4 Performance incident analysis
SAP ST03N can be used to trouble-shoot system/application performance issue by comparing system/application performance:
- between the period when performance is bad and other periods when the performance is good.
- between the user(s) who is complaining of bad performance and who is having good performance.
This performance comparison can help you to pinpoint why a program/transaction is having performance issue or help you to identify the direction of performance trouble-shooting.
Our ECC system is feeding SAP “BW” system. Once there were many idocs piled up in ECC side waiting to be sent to BW system. Using ST03N, comparing the period when performance was good with the period when performance was bad, it was found that BW side was slow in loading ECC IDOCs. Further investigation found that this was due to Oracle compression bug.
5 Further information
This post is mainly from my understanding and my work experience. It is impossible for me to list all types of performance analysis we can do with SAP ST03N… Someone might find response time distribution view of ST03n is useful, one of SAP system performance key indicator is to measure percentage of dialog step which are able to complete in single seconds. My next post on SAP ST03N is to talk about ST03N RFC profiles, for example, how to understand information of various ST03N RFC profiles.