Category Archives: Performance introduction

General Introduction on SAP performance

SAP Parallel Processing Introduction

Parallel processing is to use “divide and conquer” strategy to increase business process throughput and cut processing time by engaging more system resources. This post would take about SAP parallel processing and cover:

  1. What is parallel processing?
  2. What is pros and cons of parallel processing?
  3. What are driving factors to implement parallel processing?
  4. What influence performance of parallel process?
  5. Technical overview of SAP parallel process implementation.
    1. Parallel process solution implemented via background job.
    2. Parallel process solution implemented via SAP dialog work process.
    3. What are steps to implement a parallel process?
  6. Some standard SAP parallel solution.

1 What is parallel processing?

Let’s begin with an example, 10 customer orders are to create online in SAP system and each order would take 1 minute. There are two ways to get 10 orders created in the system:

  • You can give 10 orders to a single user and the user then create two orders one by one sequentially.
  • You can divide 10 orders into two packages equally and  ask two users to work on one of two packages to create order one by one at the same time.

The former is called sequential/serial processing and the latter is called parallel processing. In case of parallel processing, we divide the workload ( to create 10 sales orders) on hand and we engaged more resources(2 SAP work processes through 2 users ) to do the same action( to create sales order) at the same time, This is what parallel processing normally mean in a SAP context – a SAP parent program would automatically divide the workload into smaller packages based on packet size which can be predefined or calculated dynamically, then start a number of child processes to process those packages concurrently, each child process would process one package at one time, when one child process finishes processing the package and there are un-processed packages, the program would launch new child process to pick up next package until all packages are processed. Child process engaged in parallel processing can be SAP dialog or background work process in a SAP system.

Let’s continue our example. When 10 orders are created via one process sequentially, it would take at least 10 minutes but this does not meet performance requirement – 10 orders should be created in 5 minutes. So 10 orders are created via two parallel processes and that cuts runtime to 5 minutes. What we should do if 10 orders have to be created in 2 minutes in this case? 10 orders needs to be divided into 5 packages equally and use 5 concurrent child processes to create order and that would finish creating orders in 2 minutes. Let’s assume that creating one order via 1 work process would consume 1 resources, then relation between the runtime and resource needed to create 10 orders and number of users (Number of child process) is showed in table – 1.

Table 1 – how runtime is related to number of parallel child process

Case 1 Case 2 Case 3 Case 5
# of child process 1 2 5 10
Run Time 10 minutes 5 minutes 2 minutes 1 minute
Resource usage/minute 1 2 5 10
Total resource( resource used/minute x duration) 1 x 10 2 x 5 5 x 2 10 x 1

From table 1, you can see that when number of parallel child process is increased, the runtime is decreased but resource usage per time unit is creased even total resource usage to create 10 orders remains the same. This is a simplified version to help you understand basic concept of parallel processing.

You can ask two users to do two different things at the same time – one is to create a sales order, the other is to create an invoice – Even these two tasks are done in parallel but this not what SAP parallel processing is referring to since different tasks (such as different business logic/operation/program) are executed.

2 What is pros and cons of parallel processing?

What is pros and cons of parallel processing? You can refer to table -1 to understand this. Parallel processing is to increase throughput and cut processing time – this is critical in many cases. To pay for that gain, more system resource is needed or used and order of processing of each package is random.

For example, using 2 child processes, it takes 5 minutes to create 10 orders in our case. That is half of the time needed to create 10 orders one by one. The cons is that system resources usage in parallel processing period is doubled what used in serial processing period in a per minute basis. In serial processing, 10 orders can be created in a fashion of “First-in-First-out” in the same time sequence they arrive. This is impossible for parallel processing.

3 What are driving factors to implement parallel processing?

The business need and technical need -The business need is that business needs to get job done sooner than later. The technical need is might be due to load contention – for example, system has more idle resource during 1st half the processing and 2nd half processing. system is overloaded in 2nd half processing. If we use parallel process and can cut time to half, this would relieve the contention on the 2nd half processing. Most of parallel solutions are developed to meet business process need.

The fitness
– The business process is suitable for parallel processing – degree of independence, dividable and type of resources engaged. Parallel process benefit would be very limited for a business process whose transactions have a lot of dependence/sequence. You cannot ask several people to create one order at the same time no matter how big the order is and how long it takes to create it since a single order cannot be further divided. A process which is doing a lot of calculations is more suitable than the one which involves a lot of database access even database management system has its setting/configuration/techniques which control parallel processing at database level.

The capabilities – The system has needed resources/configuration to support parallel processing. If your system consists of single server with single CPU, parallel processing might help little on the performance.

4 What influence performance of parallel process solution?

Assuming that a business process is fit for parallel processing, following factors can influence performance of parallel solutions from SAP parallel solution design point view

  1. The way which workload is divided.

When total workload can be “evenly” in a way that each package can be processed in same amount of time, then the time is needed to process the total workload would be reduced. If total workload is divided into 2 packages, you use two parallel processes. If one package takes 10 minutes and the other package takes 1 hour, the system would use two work processes and take 1 hour to process the total workload. But one process would run for 10 minutes and the other process would run for 1 hour – this is an uneven situation. It would be ideal that total workload can be divided into two packages but each of two packages takes 35 minutes so the whole workload can be processed via two parallel work processes in 35 minutes.

Size of package can influence performance depends on business situation. In some special situation, you need to consider resource usage of each package under a specific size to make sure that overall resource usage by all parallel processes is what the system can afford.

Parallel solution can engage SAP background process or SAP dialog process. When SAP dialog process is used, you need to make sure that size of package would not cause timeout of child process due to hitting system limit.

  1. Parallelizing degree or the number of parallel processes allowed.

    Parallelizing degree – This is number of SAP work processes which can be engaged at the same time to handle workload. Higher degree means more SAP work processes for parallel processing – so more package can be processed at the same time, so the workload can be processed faster – this is what table-1 tells us. How many parallel processes a parallel solution should use? This depends on available system capacity and business performance requirement normally.

    The typical system resource used in parallel processing is CPU power. Each parallel process would need to use a CPU. Number of parallel processes used in a solution should not be more than number of CPU which a system has assuming that parallel process is the only program running in the system. In a SAP ERP system, there are normally many activities going on at the same time, this would reduce available CPU power for parallel processing. In a SAP ERP system, all business transactions would involve database/IO operation, during database/IO operation, it would not use CPU power of application server, this would allow you to use more parallel processes than number of CPUs which your application servers have.

    Business performance requirement is important to determine how many parallel processes you need. If a system resource is not enough to support the parallel processing, then resource has to be added. If system has more resources than what is needed to meet business performance requirement. Should we design the solution to use all available resource and let the parallel solution to finish as soon as possible? This depends. I normally prefer to give just enough resources to meet business performance requirement.

    In addition to CPU, if your solution is using a lot of memory, then you need to consider whether server memory can be a limit factor for parallel solution.

    In reality, database operation is a significant portion of many SAP transactions/programs runtime, more parallel processes would have a higher chance of creating “wait” situation in database operation. Performance benefit with increased parallelizing degree would go down and while cost would go up after a certain number of parallel processes. Parallel solution is going to amplify risk which a non-parallel solution might have. Running one copy of bad code might be ok to the system but running 10 copies of the same bad code might bring a system down.

  2. Order in which packages are processed.

    This is important when number of packages for processing is bigger than number of SAP parallel work processes allowed and processing time of each package varies a lot, the overall processing would be shorter if packages which take more time are processed first.

    For example, every package needs 1 minute and 2 parallel processes, then it would take 50 minutes for 2 parallel processes to process 100 packages. Now, imagine one specific package out of 100 packages would need 100 minutes, if this package is processed as 1st package or last package, what difference would the order difference make to the time needed to process 100 packages? It is 149 minutes against 100 minutes.

There are other factors which can influence performance of parallel processing like type of server, IO system and database etc. but these are not unique to parallel processing. Last not least, characteristics of business solution itself can impact performance of parallel solution. But we cannot go around itself if we need to use parallel solution to increase the performance.

5 Technical overview of SAP parallel process implementation

All works in SAP system are executed via various SAP work processes. An execution of a SAP business transaction/program always starts with a SAP dialog or background process and engages other types of SAP work process whenever needs arise dependent on the nature of underlying SAP program. All business logic of a SAP program is executed via SAP dialog or SAP background work process. So SAP parallel solution is implemented via SAP dialog work processes and/or SAP background work processes. If there is no free dialog/background SAP work process, no user can execute any program online and no background job can start.

5.1 Parallel process solution implemented via background job

Every package of workload is processed by one SAP job/SAP background process. But more than one job which is running the same SAP program is processing packages at the same time.

The total workload to be processed is divided into smaller packages which are represented by program variants in SAP. Then more than one copy of the program is executed concurrently in background – each works on one package via one of aforementioned variants. It is the parent program logic to divide the workload into smaller packages based on predefined criteria, then loop through those packages, keep launching child job to process those packages while SAP background process is available and there are unprocessed packages. The standard SAP Function modules like “Job_submit” etc. in SAP function group “BTCH” are used to create, schedule and monitor child jobs. If you are interested in how to program this, please check SAP document – SAP programming with background processing system.

How many SAP background processes can be used to process those packages? This can be controlled by program and system setting

  1. A predefine number of parallel SAP job/background process which a business process is allowed.
  2. A SAP server or a SAP job server group for your parallel solution – then all available SAP background processes in the SAP server or SAP job server group can be used for parallel processing. If the server group contains all servers in the SAP system, then parallel solution can use all available SAP background processes from the SAP system. SAP transaction SM61 can be used to maintain SAP job server group
  3. Whole SAP system – if you do not use a predefined number, a server or a server group, then it can use all available SAP background processes.

When you design this, please bear in mind the points discussed related to parallel degree.

So how many SAP background processes would be actually engaged when a parallel solution is running. This depends on how many packages and smaller of two values: allowed parallel degree and number of free background processes. A work process is free mean it is not engaged in any work and ready for new task. For example, assuming allowed parallel degree is 10 background processes, if number of packages is 2, you can see up to 2 active SAP jobs at the same time in SAP job monitor transaction SM37 or up to 2 running background processes in SAP process monitor transaction SM50/SM66. If number of packages is 100, you can see up to 10 active SAP jobs or SAP background processes at the same time via SAP SM37 or SAP SM50/SM66. In ideal situation, the parent program should launch 10 jobs then launch next job whenever one of 10 job finishes and there are outstanding packages.

In SAP IDOC processing or SAP interface, background processes can be launched by SAP automatically due to “volume” or “communication” failure or shortage of SAP dialog work process. When this happens, this is normally related to system or interface setting which is control via SAP transaction WE20 and SAP transaction SM59.

5.2 Parallel process solution implemented via SAP dialog work process

Every package of workload is processed by one SAP dialog work process. But more than one SAP dialog process which is executing the same SAP program is running to process packages at the same time.

Even though the idea is similar to parallel processing via background job through SAP background work process, but there are technical difference –

  1. Each package is passed to child process via program internal tables/variables instead of program variants which can be maintained via SAP transaction SE38.
  2. Child process is started via SAP asynchronous RFC (Remote Function Call) instead of SAP background job scheduling system. Please note term “Remote” – not mean that Function module called (executed in child process) is always executed in different server/system. In parallel solution, The function module is always executed in the same server/system where the parent program/process is running
  3. Each Package is processed via a SAP dialog work process instead of a background work process.

SAP dialog work process is subjected to “timeout” setting which is specified via SAP system parameter rdisp/max_wprun_time while SAP background work process has no such limit. From SAP programming point view, parallel process is implemented via CALL FUNCTION statement similar to what you seen in figure 1

Figure 1 – Asynchronous RFC call for parallel processing

Figure 1 is a piece of standard SAP code. It is from standard inbound IDOC processing program RBDAPP01. The key words for asynchronous RFC are “CALL FUNCTION“, “starting new task” and “DESTINATION“. So if you see three key words in a SAP ABAP statement, you know the program is using asynchronous RFC call for parallel processing. If you are interested in how to program this, please check SAP document – Implement Parallel Processing.

How many SAP dialog work processes can be used to process those packages? This depends on specification of “DESTINATION” and configuration of destination as well as free dialog work processes of the destination.

  1. If the ABAP program specifies a destination “NONE” – this means that the program would use free dialog work process on the server where the program is executed. So Number of dialog processes can be used is up to the free dialog process in the server. If the system has only one server, then it is whole system.
  2. If the ABAP program uses “group” destination like figure 1 – then number of dialog processes can be used is determined by RFC server group configuration (determined by SAP program parameter P_rfcGR in figure 1). SAP RFC server group controls which server of a system can be used for parallel processing and how many processes can be engaged from each server of the sever group. This is the typical parallel solution setting. RFC server group is maintained via SAP transaction RZ12.

So how many SAP dialog processes would be actually engaged when a parallel solution is running. This depends on how many packages and smaller of two values: allowed parallel degree and number of free dialog work processes. You can review parallel process via SM50/SM66. But you could not review them via SAP transaction SM37 which is for background job. However how many parallel processes you can actually see in SM50/SM66 might be different from what are actually active since SAP dialog work process are shared and a program can be rolled out of a dialog work process during waiting.

In SAP IDOC processing or SAP interface, EDI/ALE partner profile setting can have an actual effect of parallel processing. When the partner profile setting for a message has a real-time or “immediate” processing setting for an inbound IDOC processing, when many IDOCs are arrived at the system in the same/short time, each IDOC would be processed via one dialog processes subject to availability of free dialog process in the whole system, this kind of “parallel processing” can bring down system when volume is high.

SAP has provided utility to execute current SAP program in parallel with minor configuration. This is enabled via SAP transaction WLCPAR and WLCPARC. This might be handy when needs arise.

5.3 What are steps to implement a parallel process?

With above knowledge on parallel processing, it is time to discuss what are the steps to implement a parallel process:

Figure 2 – Parallel process implementation steps
Quantified performance requirement includes what volume should be processed and duration which the volume should be processed.

Design or code for a program which used in parallel processing should be well tuned before a parallel solution is considered. Parallel solution is going to run the program in more than one copies. Any expensive operation would be amplified in parallel solution. Sometime, a solution can meet performance requirement without engaging parallel solution by using alternative design or coding,

Design of parallel solution
includes implementation technique (should we use SAP background work process via job scheduling or should we use SAP dialog work processes via asynchronous RFC or both ), criteria of divided workload or the total volume (should we based on date-time, SAP organization unit, number range, number of business objects etc), packet size determination (fixed packet size or dynamic calculation of packet size), package process order (if this matters), parallel process degree ( how many child processes is enough), server group ( should server group be used or not). RFC server group or Job group itself is not within program design scope – it is a joint decision among business owner, solution technical owner and performance capacity guy. Parallel degree should consider “available system capacity” when the parallel solution hits the system.

One factor to influence whether we should use the SAP background process or SAP dialog process is how long the child process would run. Dialog work process is subject to timeout setting.

Since business and system is very dynamic – it is good that packet size and server group etc. parameters should be flexible and easy to be maintained whenever there is a need instead of hard-coded somewhere.

Validate parallel solution is to check parallel solution works well, meets business performance requirement, verifies resource usage and confirms there is no negative impact on the system which it is executed.

6 Some standard SAP parallel solution

Following is a list of SAP functions which I encountered with parallel function.

  • Inbound IDOC processing – SAP inbound program RBDAPP01, RBDMANI2.
  • SD delivery creation – SAP transaction VL10A etc.
  • SD billing creation – SAP program RV60SBAT.
  • SAP workflow processing – SAP transaction SWEQADM_1.
  • TRFC/QRFC processing – SAP transaction SMQS or SMQR.
  • SAP MRP function – SAP program RMMPS000 and RMMRP000.
  • SCM model integration – SAP program RIMODINI, RIMODAC2 etc.
  • SCM CIF comparison/Reconciliation processes.
  • BW data load and activation and reporting
  • GATP Backorder processing.

Except SAP program RV60SBAT and GATP backorder processing, all other functions use “dialog work process” for parallel solution.

7 Further information

In this post, SAP transaction RZ12 and RZ61 are mentioned. I plan to write a post on how to define RFC server group via RZ12 in the future. You can click link SMQS and SMQR for more information on how to use those transaction for performance analysis and tuning in this website. If you would like to know how do we analyze/verify performance of an existing parallel process. I would cover this in my future post.

SAP table buffering

In a SAP system, there are many layers of data buffering – IO subsystem, database system, application and program. All buffering is to make data available in memory so request for the same data can be fulfilled via memory. This helps data retrieval performance and underlying business transaction or program performance. In this post, I would talk about SAP table buffering at SAP application level. I would talk about

  1. SAP table buffering introduction.
  2. How do we determine whether a SAP table should be buffered or not in SAP development or production support?

1 SAP table buffering introduction

1.1 SAP buffering concept

 

SAP table buffering is to store data of a table in an application server memory. After data is in memory and no data change, any request for data in the buffered table by any program or business application executed in the server would be fulfilled from the memory of local application server without involvement of database server. ABAP trace in my SAP environment shows that getting data from buffer can be up to about several hundred times faster than getting data from database server.

Figure 1 SAP buffering Overview

SAP buffering pros and cons

Pros

Further clarification

Cons

Further clarification

Speed up data retrieval process

This can speed up corresponding related program and transaction.

The same data would store on memory of all application server – redundancy.

Memory demand would be higher

Reduce database load

Database server is a Central Instance

Change synchronization delay.

Data retrieved by program can be out of date

Reduce network traffic

 

Buffer bypassing can happen.

Buffer is not utilized

Reduce process residency time in a SAP work process

Reducing possible contention on work process resources

   

Figure 3 SAP Table buffering pros and cons

1.2 SAP buffering types

SAP allows three type of buffering choices – single record buffering, generic records buffering and full table buffering. All those types of buffering are referring to size of “data entry” which would be managed by the SAP buffering mechanism in buffer loading, swap, data change management.

 

Buffer load during read when data is not in buffer

Buffer change/invalidation

Buffer swap

Single record buffered

Load one record into application buffer at one read. A record would not be in application buffer if it is never read since start of the instance

One record at one time.

Swap would be done at record level.

Generic area buffered

Load all records which share the same partial primary keys. No read means not in buffer

All records which share the same partial primary key of the changed records are invalidated at the same time.

All records shared the same partial keys are either swapped or in memory as a whole.

Full table buffered

Full table is loaded into buffer at 1st read

Whole table are invalidated upon any change on the table.

Whole table would be swapped.

Figure 4 SAP table buffering types

In following section, I would talk about how to select right buffering type for a table.

1.3 SAP buffer for table buffering

The application server memory used for SAP buffering is called SAP buffer. There are two types of SAP buffer used in table buffering – Generic Key buffer and single record buffer. Generic buffered table and fully buffered table would share the same SAP buffer – Generic Area Buffer. Size of SAP buffer is configured by two parameters: one is to specify size of buffer,the other is to specify number of the directory entries in the table buffering.

 

Buffer size parameter

Further info

Number of entry parameters

Further info

Generic area buffer

zcsa/table_buffer_area

Buffer Memory size

zcsa/db_max_buftab

Maximum number of buffer that can be buffered

Single record buffer

rtbb/buffer_length

Size of single record table buffer

rtbb/max_tables

Max. number of buffered tables

Figure 5 SAP tabe buffer parameters

1.4 SAP buffer synchronization

After a SAP table is buffered, the buffered data can be changed, this data changes need to be reflected in generic area buffer or single record buffer of all SAP servers in the system.

Any change by a sap program/user is always initiated from an application server. On the server where change is initiated, the table/data is marked “changed”/”invalid” (Actual state is “pending” in ST10) as when buffered data is changed (Whether change is committed or not makes no difference based on my testing). When buffer table is marked as “changed”/”invalid” ( “pending” status in ST10 ) in a server, the data request from the server would be fulfilled by database until certain number of read operations on the table occurs from the application server. After that number, buffer would be refreshed and marked “valid”. This mechanism is to reduce load from pre-mature buffer reloading or buffered being refreshed too often un-necessary. For example, a program can update a table hundreds or thousands of time in a seconds, if the system is going to reload the data into buffer for each update, this could bring down the systems.

Now what are footsteps used by other servers for buffer synchronization? When a buffered table is changed, a change entry is logged into SAP table DDLOG. SAP buffer has a mechanism to read DDLOG table “PERIODICALLY” to know what buffered table is changed and from which server, based on that information, it would mark the buffered table as “invalid”/”changed” in all remaining SAP application servers. So there is a delay between data is changed in one server and corresponding data in memory is marked as “invalid”/”changed” in remaining instances of the system. Extent of delay is determined by parameter RDISP/BufRefTime, timing of a data change, volume of buffer table changes and instance/server performance. After a table is in “pending” status, data needed by a program from a server would be from database table. The table would be loaded into buffer again after a certain number of read operations on the server. Please click the post how ST10 buffer state is change for details. So remember when table is in “valid” status, the data would be read from buffer for the server even it is already changed in another server!

Apparently, the delay of buffer synchronization only matters for the SAP system with more than one application server/instance.

What are SAP buffer synchronization parameters? They are:

  • RDISP/BufRefMode – when there is more than SAP application server/instance, this parameter should have value of “Sendon, ExeAuto” – this enables buffer synchronization. Value of “sendoff, exeauto” means no buffer synchronization – applicable for sap system with only one SAP application server/instance.
  • Rdisp/bufRefTime – this specify interval between two adjacent buffer synchronization so buffer can be synchronized periodically. It has a default setting “60” seconds.

You can use SAP program RSDBBUFF
to know when latest buffer synchronization is done for an instance/server and how far away the server/instance is from next buffer synchronization. However if you would like to know what table buffered entries are changed in the past, SAP program RSTUNE60 can help.

1.5 how to activate SAP table buffering

You might be wondering how to enable buffering for a selected table? Table buffering is a technical setting of a SAP table. SAP transaction SE13 allows you to display or change technical settings of SAP table directly. On the technical setting screen, you just need to switch on the buffering and check the right buffering type as showed in the following screen. You know you need to save the changes – after that, you are done – your table is buffered as per your instruction.

Figure 6 SAP table buffering setting

When you select buffering type – Generic Area buffered, you need to specify how many key fields should be used according to your buffering design. Of course, number if key fields specified should be less than number of fields which the primary index has… otherwise, it is a single records buff.

You can use SAP SE11 transaction to display and change technical settings using menu option of SE11 screen( 2nd screen) as well

1.6 SAP buffering monitoring

SAP transaction ST02 can be used to monitor SAP buffer and memory at instance level to see whether there is enough buffer space. SAP transaction ST10 can be used to monitor access status of both buffered SAP tables and not-buffered SAP tables.

1.7 Table buffering makes a performance difference – an example

Following is result of a ST12 ABAP trace on a business transaction execution. TABLE USR05, TVTA and /VSO/M-VSO_SETUP are buffered configuration tables. VBAK and VBUK are transaction table which has no buffering. Select statements are using full primary keys of those tables.

 

 

 

Table name

Buffer status

Total # of read

Total time(micro seconds)

Time per read(rounded)

TVTA

buffered

58

826

14

USR05

buffered

1,474

26,201

17

VBAK

No

2,342

8,260,148

3,526

VBUK

NO

2,342

6,769,828

2,890

/VSO/M_VSO_SETUP

Buffered

2,342

23,370

10

 

 

Above table shows a significant SQL runtime difference between reading buffered table and non-buffered table when primary keys are used.

2 How do we determine whether a SAP table should be buffered or not?

2.1 What should we consider to buffer a table?

Bye now, you know how SAP table buffering is working now and significant runtime improvement for table access through buffering. So you might wonder how we can reach a decision on whether a SAP table should be buffered or not.

To reach decision, you need to consider following factors/guidelines

Figure 7 SAP table buffering – influencing factors

Following are further prints to help you to understand above points better:

  1. Table size – In general we should not buffer a big table since memory is normally limited. We should be extra diligent when you buffer a table which is bigger than 1% of defined buffer space.
  2. Data change habit – This refers to when, how often, volume of change related to table. In general, we should not buffer a table which is changed frequently. A table with a change/read ratio > 0.5 should not be buffered normally.
  3. Data access habit – This refers to access frequency, access time pattern, access volume pattern (associated selection criteria) and load generated by the SQL. we should not buffer a table which is seldom accessed/queried by SAP programs/transactions. How table is access or access pattern is important as well. Buffer a table with higher database load can be good for database performance. Common wisdom is what is good to database would be good to SAP application. Your option to buy into thisJ. Certain SQL SELECT statement would bypass buffer. If majority of SQL statements being used for a table would bypass buffer, then buffering the table makes no sense.
  4. Buffer synchronization delay impact – We should not buffer a table when buffer synchronization delay results in obsoleted data being read by other SAP programs/transactions. This is not a concern for a SAP system with only one application server. If business insert/update data in a quiet window (Data change habit), then this is not a concerns any more.
  5. Business performance requirement – If buffering a table can improve a business process performance significantly, then we should consider buffering seriously.
  6. Buffer and memory status – If buffer space is not enough and cannot be increased due to physical memory limitation, buffering new table might create more displacements or swaps. If there are heavy swaps in table buffer area, you would make the things worse to buffer additional table, I think you can agree with me on this. It is better to maintain 10% of free buffer space – do not forgot to check free directory as well.
  7. Database server and application server load status – this would help us from load distribution point view – where is the best place to keep the load based on specific case. The general wisdom is to remove load from database server to application servers. but if your application server is already overloaded and database server has a lot of idle power, it would not makes sense to buffer a table before application server load can be mitigated. The normal recommendation is that SAP server/instance should maintain at least 20% free CPU power for peak processing – this is especially important to DB server.

Standard SAP table has a predefined setting on table buffering attributes. You should not change the setting for a standard SAP table in general. A SAP tables which stores business transaction data cannot buffered. SAP tables for master data are not buffered. SAP tables for configuration table are normally buffered. However, buffering setting on standard SAP setting can be changed suited to your specific business needs.

Consultants from SAP recommend that table over 5 MB should not buffered. But if this table is frequent accessed, very expensive, very stable, there are enough free buffer space and buffering the table can help program to meet business performance goal, then buffering the table might be a wise choice even it is 10 MB in size. If a table is small and seldom changed, but there is a high chance that delay of synchronization can create data inconsistency and impact accuracy of business result when there is a change, this table cannot be buffered.

Table buffering type is a fine setting mainly dependent on table size, table change habit and access habit. The setting would impact size of data which would be marked “changed”/”Invalid”, If each time, only one record is changed:

  • Fully buffered -> all records of the table would be invalidated and reloaded to buffer by the system. One change would result in loading of whole table.
  • Generic buffered ->all records which share the same common keys fields would be invalidated and reloaded to buffer by the system. Access other records in the affected group needs to get from database after they are marked “changed” until they are reloaded. Risk of getting obsoleted data if data is accessed after change and before synchronization.
  • Single record buffered -> only the change record is invalidated. Other records would be valid in buffer. But single record buffered needs more round trips between database and application server to buffer a table.

Under fully buffered and generic buffered, I would like to highlight that change one record would cause more than one record being reloaded to buffer from database. When table size is big, volume of change and access is high in the same time window, this can create serious system performance issue due to frequent buffer reloading- I have seen this in my work.

If many records are changed each time, “full” or “generic” table buffering might be working better to avoid too many round trips between application server and database server related to buffer synchronization. Number of key fields should be decided based on fields reference in SQL where-clause.

So what tables can be buffered or cannot be buffered – you need to review all those factors and understanding what is your key driving factors based on your business need. There is no hard line here. Bottom line is that you need to know the context surrounding the table buffering – the system, the business application and the table.

2.2 How do we know whether a SAP table should be buffered in SAP development/project?

Here, we only consider “new” local table developed in a SAP development. Designing a new local table involves many areas – like table structure, primary key and index, data retention, data management etc. Among them, it is buffering attribute. If you do not specify buffering attribute explicitly, the table would not be buffered by SAP. Many developers overlook technical settings of SAP table. In the end, it creates performance issue and more effort to correct this later.

Transactional table cannot be buffered normally. We only need to consider local table like mapping, reference and set up tables by collecting needed information related to previous stated table buffering guidelines:

  1. Who would populate data? Then when/how frequent is the data inserted to the table? How many entries would be inserted to the table? Who can be a person, a SAP job, a SAP program and an interface.
  2. Who would use the data? Then when/how frequent is the data accessed? What selection criteria would be used to access data? How many records would be accessed by each user.
  3. Who would modify the data? Then then when/how frequent is the data modified? What are selection criteria for data, how many records would be updated at one time window.
  4. How long would data stay in the table? This is data retention period.
  5. Business performance requirement. If program performance is critical, spends notable time on this table and there is no way to optimize the access, then buffering might be important.
  6. Production server status – buffer/memory status and Database server/application capacity status.

Based on table definition, you know table record size already, then based on above information you collected, you can get a good indication about table size. With change frequency and access frequency, you can get a good indication about change ratio – # of changes/# of reads.

With above information, you can reach decision on buffering. For example:

  1. Small table and seldom changed can be buffered if servers status(buffer and CPU) has no issue.
  2. Table is not small but it is access a lot – table in general should be buffered if change habit is not a concern.
  3. Table is small but change habit is a concern – then this table cannot be buffered generally.
  4. Table cannot be buffered if change habit would cause data inconsistency which business cannot afford. If data is changed well before they are being used, then buffer synchronization delay is not concern anymore.

Changes here include data population, data modification and data deletion. If data is changed well before they are being used, then buffer synchronization delay is not a concern anymore.

In SAP project/development, you might need to buffer an existing local table as well, this is mentioned in following section.

2.3 How do we know whether a SAP table should be buffered in SAP production/support?

In SAP development, most information for table buffering is from business requestor who requests the development. In production box, we have different SAP tools to record table related information. We can get a lot of information related to table size, access habit, change habit and system status information.

The best starting point to analyze table buffering is to use SAP transaction ST10 – table call statistics. ST10 analysis can provide a list of table which should be reviewed here further. Click here to know how to run SAP ST10 to do sap table buffering analysis.

If SAP ST10 states that a table can be buffered, then you need to do more investigation to confirm whether it can be buffered.

  • Get table size – normally refer to disc space but can be refer to number of entries. SAP transaction DB02 can tell you disc space and history. SAP transaction DB20/DB20ORA and SE15 can show you number of entry in a table. DB20/DB20ora report number of table entries from oracle statistics table while SAP SE16 transaction tell you accurate number of entry in a table by accessing each record of the table.
  • Understand how the table is accessed by analyzing table SQL statements Cache via SAP transaction ST04. SQL cache can tell how many seconds the SQL has spent so far, type of SQL operation and total of number of operation. For select/update statement, it can show you the where-clause which you can get clear picture on select-conditions.
  • Run ST02 and ST06 – check table buffer status to see whether there is any contention here. Using ST06 to check CPU status on database server and application server as well as main memory to see whether there is potential concern related to buffering.
  • You need to talk with business owners to understand remaining details of access habit and change habit as well as possible impact from delay of buffer synchronization.
  • You also need to understand the performance requirement – based on current status and time spent on that table, you can know what runtime improvement you would get by buffering the table.

With above information, you can make an informed decision on table buffering. Following the similar process, you can confirm whether you should stop buffering on a table or change buffering type of the tables. Again, ST10 information like number of invalidation and number of database records have several contributors – like se16 transaction would bypass buffer and increased number of rows. It is not safe to make a decision to remove buffering on a table just based on ST10 number of invalidations and number of rows impacted.

3. Further clarification

SAP table buffering does not support secondary index. If a buffered table is big and reading the table is not using primary index, then read a record from such table is consuming more application server CPU and benefit of runtime improvement might not be that significant than read it from database using secondary index. You might need to test to see the improvement before buffer and after buffer to make sure that runtime improvement worth the consumption of CPU and memory in application server side.

SAP application buffering also include number range buffering. Number range buffering is important for application performance. However this is not covered here. I might cover this in future posting.

There are other types of SAP buffer like program buffer etc., those are important to SAP application performance as well. They are configured by SAP basis team. I have covered them in my post – how to run SAP ST02 and use ST02 to do performance analysis on SAP memory and buffer.

We should not confuse SAP buffering with program level data buffering. Data in SAP buffer is available to all SAP programs while program level buffering is only available to a specific program while the program is running normally.

I assume that you have known how to run SAP transaction ST10, SAP transaction ST06, SAP transaction ST04 and SAP transaction DB02. I plan to do a post on ST04 and DB02 in the future.

SAP Performance Introduction

Content

Definition
Why performance matters
What influences performance
How can we achieve good performance
Performance issue classification

Definition

When we think of performance of application or system, the first thing which comes to our mind is time the time which a business transaction or system take to complete business work. Time is money. So there is no doubt that time is a critical measure or characteristics of performance. If a business transaction cannot finish in business expected time, the performance is not good for sure. Would it be true that performance is good if a business transaction can always finish in time? Is time the only measurement or characteristics of performance?
My answer to both questions would be NO.
A transaction can finish in expected time does not necessary mean the performance is good. An individual transaction can complete it works in time but at expense of system resources or other programs performance. It can use more CPU, memory, network etc resources than what it is really need for the job. By doing so, it could take resources away which otherwise is available for other functions and create extra demand on resources which might need more investment. By taking away resources, it can impact other transactions which are running at the same time. In a SAP system, there are many resources you can use different resource combination to get the same work done. I see it often that cutting down resource usage can reduce time which a transaction takes. It is not a surprise resource usage is always associated with time. Resource usage is transparent to business users. It is a technical concern which would ultimately impact business.
Performance is also concerning about citizenship in its operating environment. Many business functions and processes share the same operating environment. Running one function or process should not impact following colleagues in the operating environment. A transaction which has a good performance should leave footprint in the system as less as possible w/o impacting its follow transactions in the system.
So performance of a business process or transaction or function should have following characteristics in my review:
Time – Its abilities to meet or beat quantified business performance requirement in a normal system status.
Resources – Its abilities to use system or application resources as much as it needs, as when as it is necessary and in a balanced way.
Citizenship – Its abilities to be transparent to other existing/future processes/functions in the operating environment.
Now, it is obvious that good performance in my position is that a program can meet quantified business performance requirement, use as much resource as it needs, user resource as when as it is necessary and have no impact to others functions in the SAP operating environment which it is run.
Have a good understanding of performance is the basis for performance analysis and performance measurement. Having SAP project management and business stake holders to have a good understanding on performance is the key to get needed support.

Why performance matters

Good performance helps to maintain company reputation which is valueless. Imagining what damage would be done to a company if the company cannot produce financial report in time due to system performance.
Good performance makes everything predictable, this helps business to do planning and coordination. Order management process owner can know how long it takes to get order loaded and how long it takes to send order to warehouse for picking, when everything can be ready for shipping etc.
Good performance normally means more work can be done in the same time. So employee can become more efficient and this can increase their job satisfaction.
Good performance can also help to lower solution TCO. It allows more to be done using existing resources. It helps to avoid un-necessary hardware upgrading. It lowers operation support/maintenance cost. It helps to give employee positive job experience and job satisfaction.
Good performance also can help to improve customers satisfaction and help to retain customers. Customers might go to different supplier if you cannot ship their orders in a timely fashion.

What influences performance

I review performance as a product of system which has many components. In my view, a given SAP performance system has following key components
o System SAP System here is referring to IT infrastructure and configuration of IT infrastructure where application and program is operating in. This includes but not limits to server, network, storage etc. It is obvious that we would not have good performance if related IT infrastructure is not enough or it is not appropriate. Some components of IT infrastructure like network are shared within with other applications. When we talk performance, we need to remember status of IT infrastructure is important to performance as well.
o Application This is referred to SAP application package and its architecture which is delivered by SAP company.
o SAP configuration This includes SAP system resource allocation, database configuration and SAP basis configuration etc.
o Business solution This includes solution architecture, all development/program or configuration made in SAP for a business solution and the deployment of the business solution.
o User – I am mainly referring to end business user who would executes/owns a SAP business transaction in the system. Business user is ultimate driver for the system, application and programs. It is based on their input that a system is built and measured. I am not referring to the IT professional who design, build, deliver and maintain such system, application and programs.
Performance issues can root from each component of a SAP performance system in general. But Most of performance issues are attributed to business solutions. Based on my experience, I rank them in following order based on frequency where performance issue happens.
Business solution Majority of performance issue is related to local development. In my experience, 70% of performance issue is related to business solution.
SAP configuration SAP configuration can cause a performance like no enough memory, no enough work-process etc.
Users Users can cause performance issue in the way on how a transaction is executed. Performance of a transaction can varies based on user input to the program.
SAP Application I have seen it again and again that performance issue can originated from standard SAP solution due to special scenario of your processes such as high volume etc.
System System can creates performance issue especially when network is busy, file-system is out of space, or there is an IO hotspot etc.

What we can do to have good performance

Good performance of a SAP application is a result of total application performance control in SAP development/project or/and proactive SAP performance management in operation.
In SAP development/project, we need to have an end-to-end performance control approach – which build different performance control activities into each phase of SAP project/development supported by performance tools and performance processes. There are different ways to check and verify performance at different stage of the project. Different ways complement each other and work together to ensure that project solution has a good performance. It focuses performance issue prevention, earlier detection and earlier resolution. This helps solution to meet performance objective and help project to deliver on schedule.
Proactive performance management in operation is to proactively monitor health of system and application, find trends of resources usage and detect potential performance threat earlier using existing SAP performance tools/transactions.

Performance issue classification

There are different ways to classify performance issues. I would cover some of them which I think important.
Performance issue can be divided into run-time issue or non-runtime issue. Runtime issue means that user has to wait longer than normal to get systems response. Non-runtime issue is referring to the situation like program abortion on resource contention like lack of memory, out of disc space etc.

Base on impact scope, performance issue can be classified into:

  • System level – System level performance issue would impact every function and every user in the SAP system.
  • Instance level – A system can have more than one instance. Instance level issue would impact every function and every user in that instance.
  • Site-level – Site level performance issue would impact every users on one particular site and office.
  • Business function area level – Function level issue is refer to performance issue
  • Process/transaction level – This is the most common performance issue reported by user. This issue is normally on individual
  • User level – only single user is reporting performance issue – like performance issue at presentation level or local pc level.

Based on technical nature, performance issue can be classified into:

  • Program/code issue Performance issue is rooted from the program itself including design and code.
  • Local system resource issue Performance issue is rooted from shortage of system resources like CPU, Memory and Disc-space. Memory includes SAP memory and database memory.
  • Local system issue Performance issue is rooted from system/database status like system and database bug etc
  • Remote system/client issue Performance issue is rooted from external system/client like remote system/client is not available or too busy.
  • Network issue Performance issue is rooted from network issue like sudden increased network traffic etc. SAP server and non-SAP server are working as normal.

Understanding scope of performance issue is important to assess priority of the issue and possible cause of the issue.

Further, we need to clarify a performance issue: whether it is one time issue or it is a consistent issue. The former might be a treated an incident, normally have quick solution can be related to random system status or wrong operation of user, improper input etc. The latter is a problem which might be rooted from design, program code or consistent business pattern change etc and need more effort to address this.