Explore options view

Define the explore method (action-based, request-based, or both) that AppScan will use to explore the application, as well as other basic and advanced explore settings.

Explore Method
AppScan uses two distinct methods for the Explore stage of the scan. You can select either one, or both. Of the two methods, Request-based Explore is usually faster than Action-based Explore. When both are selected (this is the default and recommended selection), Action-based Explore runs first, with a 30 minute time limit, followed by Request-based Explore.
Page Structure (DOM) Filtering
These can greatly reduce scan time by identifying pages that are similar enough to pages already scanned, that they can safely be ignored.
Scan Limits
These determine how deeply (or how quickly) AppScan explores your application.
Advanced Settings
  • Additional explore settings: For configuring the client to recognize a specific server encoding and to send a specific user-agent header.
  • Action based: Settings specific to this method.
  • Request-based: Settings specific to this method.

Setting

Details

Explore method

Action-Based

A version of the Google Chrome browser is used to scan the site, as a user would, clicking the links that are visible in the browser. This method is particularly effective where technologies such as JavaScript and Session Storage are used, and for sites that are RIA, Single-page Application (SPA), or AngularJS.

Request-Based

Requests are sent based on all page content that AppScan discovers. This includes content that is not visible to users using a browser, such as links in comments, which an attacker would find.

Page structure filtering

Filter similar pages based on structure (DOM)

AppScan® compares new pages with those already scanned, for structural (DOM) similarity, which indicates the new page contains no new links or content that require additional testing. For example, on a commercial site there may be a catalog with individual pages for a thousand different items, that are in all other ways identical. There is usually no need to scan all those pages. Filtering based on DOM similarity can greatly reduce scan time.

By default both check boxes are selected. After the scan you should examine the Filtered tab of the scan results to see whether unique requests were mistakenly filtered out of the scan. If this happened you should try the "Filter less pages" option, which maintains a steady, lower level of filtering, or disable DOM filtering altogether.

Three kinds of filtered items will be found in the Filtered tab of the results:
  • Similar DOM: This indicates a page that was filtered from the scan because its structure (DOM) is similar to that of a previously explored page, and probably contains no new elements to test.
  • Likely Similar DOM: This indicates a request that was not sent at all, because AppScan estimates that the response will have the same structure (DOM) as that of a previously explored page, and will contain no new elements to test.
  • Similar Body: This indicates a request (from a page that was not filtered due to Similar DOM) that was filtered from the scan because its response body content is similar to that of a request that was previously explored.
After the scan you should examine the Filtered tab of the scan results to see whether unique requests were mistakenly filtered out of the scan. If this happened you should clear the "Filter likely duplicate pages" option (next option), or disable DOM filtering altogether by clearing this check box.

Filter pages that are likely to be similar based on structure (DOM)

This setting filters "Likely similar DOM" pages from the scan (see description above). If unique requests are mistakenly filtered out of the scan you should clear this check box.

Scan limits

Redundant Path Limit

AppScan® will not access the same path more than the specified number of times.

A particular path may be visited several times if it appears with different parameters. This limit is relevant mainly for scripts. It is deselected by default, as in most cases selecting the check box above, Filter duplicate pages based on structure (DOM), will sufficiently control scan time.

Click Depth Limit

AppScan® will not scan pages that are accessed by clicking more than the specified number of links.

Total Page Limit

If selected, AppScan® will access no more than the maximum number of pages defined. Note that there may be many URLs explored per page.

Advanced options

This section includes three parts:
  • Additional Explore settings
  • Action-based: settings that affect action-based exploring only
  • Request-based: settings that affect request-based exploring only

Additional explore settings

These advanced settings apply to both action-based and request-based exploring.

Setting

Details

Encoding

AppScan® generally detects the application's encoding method automatically, and therefore Autodetect is selected by default.

If the content of responses in the scan Results looks distorted, this may mean that the encoding method was not correctly identified. To solve this problem, select the correct encoding method from the drop-down list.

User-Agent

The user-agent header in an HTTP request tells the server what kind of client sent the request, and this may affect the content that the server returns. For example, there may be content that is specific to mobile phones that is sent only when the user-agent is a mobile phone browser. In order for AppScan® to be able to test such content, you need to configure it to send the appropriate user-agent header.

AppScan® generally detects the user agent automatically, and therefore Autodetect is selected by default. However, if you use a browser other than the built in browser, and you do not record a login procedure, a multi-step operation, or a manual explore, AppScan® will be unable to autodetect the user agent, and you must select it manually.

To change the user-agent, select an agent from the drop-down list.

To enter custom content, click the Edit button and type in the content. When you close the dialog box the button name changes to Custom User Agent.

Note: If you change the default browser, refer to the conditions listed in Changing the default browser

Action-based

This section is active only if the Action-based check box is selected at the top of this dialog.
Setting Details
Explore Timeout (minutes) The default time limit for Action-Based Explore of a site is 30 minutes, after this time the Explore stage stops even if the site has not been fully covered.

If AppScan misses significant parts of the site in this time, you can increase this timeout.
Minimum wait before invoking actions on page (milliseconds) AppScan attempts to identify that a page has loaded completely before it starts exploring it.
If you add a minimum wait period here, AppScan always uses this setting as the minimum wait period (even if it detects that the page has loaded), but will wait more than this time if it detects that the page has not loaded.
Tip: If when reviewing the Explore data, you see that AppScan failed to execute all the possible actions on a page, this can indicate that its dynamic wait time was too short. You can also see this during the scan if you enable the browser:
  1. Go to Tools > Options > Advanced
  2. Locate SessionManagement:ShowActionBasedPlayerWindow, and change its setting to True.
  3. Run a scan. The browser will open when scanning, and you can watch as AppScan explores your site. If you notice that it continues to a new page before the current page is fully loaded, increasing the wait time may solve the problem.
Note: Changing this setting might affect Explore time, so you may also want to consider increasing the Explore timeout (above).
Dynamic pages
Auto-detect dynamic-page loading By default, AppScan actively detects dynamic page content and treats the page as such. In rare cases this may prevent the page from loading correctly, and therefore affect scan coverage.
Tip: To identify this issue:
  1. Go to Tools > Options > Advanced
  2. Locate SessionManagement:ShowActionBasedPlayerWindow, and change its setting to True.
  3. Run a scan. The browser will open when scanning, and you can watch as AppScan explores your site. If you notice that pages which load successfully in a regular browser do not load correctly during the scan, clearing this check box may solve the problem.
Filters
Skip actions on identical DOM elements AppScan identifies actions that it has already executed on a previous page based on various criteria. If your site includes different actions that might seem identical due to their DOM element, AppScan might incorrectly ignore them. If this happens, clear this check box.
Note: AppScan does actually repeat identical actions a few times, to make verify that they really are the same, before deciding to ignore future iterations.
Use machine learning to analyze and skip redundant actions AppScan uses Machine Learning to improve Explore stage efficiency. AppScan can predict actions that are likely to lead to already-discovered parts of the site, so it can avoid them.

If your site includes many pages whose only difference is their content, this feature can greatly increase site coverage within the defined Explore Timeout. The precise gain will vary depending on the site.
Actions to skip This is a list of actions for AppScan to ignore as they could adversely affect the scan, or even the application. Actions to be skipped are identified based on the Id, name or ng-model attributes of the DOM element for the action. Any action whose DOM element attributes' Id, name or ng-model contain one of the words in the list, will be filtered out of the scan.

You can Add, Edit, and Delete items in this list.

Request-based

This section is active only if the Request-based check box is selected at the top of this dialog.
  • JavaScript™ options determine whether AppScan should ignore or scan these scripts.
  • Explore Mode determines whether AppScan® explores all links on a page before continuing to the next page, or explores each new link as it is found.
  • WebSphere Portal are for configuring the client to recognize a specific server encoding and to send a specific user-agent header.
Setting Details
JavaScript™
Parse JavaScript™ code to discover URLs AppScan® will parse JavaScript™ code as text data to collect links.
Explore mode
Breadth First (Default) AppScan® explores page by page, exploring all links on one page before continuing to the next.

It is recommended that you do not change the default selection of this option (Breadth First), unless you are aware of limitations in your application that demand that a user visits links in a specific order.
Depth First AppScan® explores link by link, exploring each new link as it is found.

If you change the Explore Method to Depth First, you must also change AppScan® to use only one thread during the Explore (in Configuration > Communication and Proxy view).

WebSphere® Portal
Enable WebSphere® Portal scanning

If the site is a WebSphere® Portal site, AppScan® will need to get URL decoding information from the site for more efficient scanning and to build a useful application tree. To enable decoding, select Enable WebSphere Portal scanning.

If the context root URL does not follow the default format, click Add Context Root URL to add one or more context root URLs.
Tip: If you are not sure what your portal's context root URL is:
  1. On the computer where WebSphere Portal is installed, open the wkplc.properties file in the wp_profile_root/ConfigEngine/properties directory.
  2. The context root value is specified by the WpsContextRoot property.
Tip: When scanning a WebSphere® Portal site, it is recommended to use the predefined WebSphere® Portal scan template, which is configured for the purpose.