soatest培训手册loadtesttraining.ppt

Parasoft LoadTestAgenda§Load Testing Introduction - Why Load Test?§Creating and performing a SOA-based Load Testing§Customizing LoadTest §Machines§Profiles §Scenarios§Monitors§Command-line execution§Viewing reports§Review infrastructure and procedures - TBDLoad Testing – First Review the SLA§Identify the criteria of the SLA§Why are we Load testing?§What are the details of the SLA?§What is being guaranteed?§E.g. # of sessions§What are the sequences in a session?§Are there performance/timing requirements?§What is the distribution of session§Build the Model in Parasoft LoadTest§Establish the distribution of Load profiles (session types)§Establish the SLA metrics (Quality of Service), e.g. timing of the session or session components, as specified in the SLALoad Testing – Validate the SLA§Process the results of Parasoft LoadTest§Results within defined metrics – success§Results outside of defined metrics – failure§Where is the problem? Hardware or Software§Use monitors to diagnose the system§Ensure ongoing maintenance of the SLA§Schedule periodic testing of the live system§Small scale tests during normal execution to ensure metrics are within defined parameters§Results go to Parasoft Concerto for monitoring§Failures go out as emails for alertsQuick Example using LoadTest Wizard§Start LoadTest:1.Create Load Test from SOAtest TST file2.Select Test Suites for Load Profile(s)3.Define Load Schedule/Scenario4.Select Monitors (optional - skip)5.Select Quality of Service criteria6.Add Load Machines (optional – skip)7.Select ‘Start Immediately’, ‘Record Individual Hits’, Click ‘Finish’Review progress during the run and report data at the endLoadTest Configuration - Machines§Define a LoadTest cluster to generate greater load§Distribute generated load across machines§Use High-throughput mode to put the system under stress (no validations)§Use ‘regular’ mode to validate system behavior§Start on Linux machines:loadtest -loadtestserver [portnumber]§Documentation: §Load Test Command Line Interface (cli) > Starting the Load Test Server on Remote MachinesTest ApplicationLT MasterLT ServerLT ServerLT ServerLT ProbeDistributes Load GenerationRequest/Response PairsValidates functional behavior when the system is under loadProvides High throughput tests to put system under loadLoad Test Infrastructure: Master & ServersLoad Test Infrastructure: Load Balancer and Server InstancesMaster(Validating Responses)LoadTestServers(High throughput load)Load BalancerServiceInstancesLoadTest Configuration - Profiles§Defines SOAtest Test Suites/Scenarios used/controlled§Control delays between invocations§End to Begin – How long after the end before start another§Begin to Begin – How long after start before start another§One SOAtest TST file used to define a LoadTest project (select Profiles folder). §What Environment is used §How data sources to be used by virtual users and distributed§How setup tests are to be run§NOTE: If you need to run tests from multiple TST files, create a TST for Load testing that has ‘referenced’ test suitesLoadTest Configuration - Scenarios§Defines how the Load test is to be performed§Control parameter Virtual Users or Hits Per Second (Tests Per Second)§Duration of the a test§Distribution over time, profiles and machines§Quality of Service Metrics (expand a Scenario)§Defines success of the Load test§Numerous categories of QoS metric§Reporting and Automated Stopping (select Scenarios Folder)§What data to record§When to stop if there is a problemDifference between VUs and HPS§Example§Avg. Service Response time = 2 seconds§Load Test Duration = 180 seconds§Scenario: VUs = 5vu§Total Transaction Count = §(duration / response time) * # of VUs§Example: 180 / 2 * 5 = 450§Scenario: HPS = 3§Total Transaction Count§(Duration * HPS)§Example: 180 * 3 = 540QoS: Execution vs. Server Time§Execution time: §Includes the time to transmit the request and receive the response §Server time: §Time from last byte out to first byte inLast byte of response receivedFirst byte of response receivedMessage in transitMessage in transitMessage in transitMessage in transitMessage in transitMessage in transitMessage in transitService is “thinking”Service is “thinking”Service is “thinking”Message in transitMessage in transitMessage in transitMessage in transitMessage in transitMessage in transitMessage in transitLast byte of request sentFirst byte of request sentUnderstanding Timing DataWebApplicationLT ServerTimeRequest Send TimeResponse Receive TimeServer TimeExecution TimeLoadTest Configuration - Monitors§Monitors system information from the infrastructure and correlates with ‘over the wire’ data§Types:§Windows Perf Mon§JMX – WebLogic/WebSphere§SNMP§rstat (requires additional licensing from 3rd party)§Custom (Java, JavaScript, Jython)§ > SOAtest > Scripting Examples > Scripting Custom Load Testing Monitors§Documentation: Using SNMP, Windows, JMX, rstat, and Custom Monitors > Adding Monitors > Adding Custom MonitorsLoad Test Infrastructure: MonitorsMaster(Validating Responses)LoadTestServers(High through-put load)Load BalancerServiceInstancesMonitors on remote machines send metrics/stats back to the MasterCustom Monitor Example§Jython Example: using java.lang.Runtime object to get the current memory status of the JVM. from java.lang import Runtimedef getTotalMemory():  return Runtime.getRuntime().totalMemory() / (1024 * 1024)def getFreeMemory():  return Runtime.getRuntime().freeMemory() / (1024 * 1024)def getUsedMemory():  return getTotalMemory() - getFreeMemory()Reports – LoadTest Binary Report§Interactive report viewable in LoadTest§Summary, Statistics, QoS metrics§Detailed interactive graphs§Filter data based on machine, profile, test§View Errors, drill into data§Export data for selected transactions§Compare results between different runs§Create composite reports with saved binary reportsReports – Defining HTML Reports§Build custom HTML report§Specify Tables and Graphs to be included§Right Mouse Menu > View Report Configuration …§View/Save HTML report§Right Mouse Menu > View Report or Save As …Command-line Automation§LoadTest command line calls command file§Command file opens LoadTest .lt file and executes the loadtestOpen FilePathloadtest [-minutes Number] [-vus Number | -hps Number] [-report reportFile.rpt] [-html htmlDirectory] ScenarioName§Documentation: §Load Test Command Line Interface (cli) > Running Load TestsReports – LoadTest Continuum§LoadTest Continuum = Concerto plugin§Command line sends data to the LTC§Select Specific project§click refresh if project not available§Stores History, HTML and Binary reports for each Test。