tag:support.arachni-scanner.com,2012-07-01:/discussions/questions/13045-how-to-configure-arachni-to-be-able-to-scan-pages-which-contains-navigation-based-on-tab-pathsArachni: Discussion 2016-11-28T15:24:37Ztag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-17T13:05:03Z2016-10-17T13:05:03ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Regarding the <code>--scope-extend-paths</code> option, it
doesn't work with URL fragments yet.</p>
<p>About the rest, I can't tell what's going on without access to
the application, although Arachni mostly works on events and it
should have clicked the tabs.</p>
<p>Have you tried the <a href="http://downloads.arachni-scanner.com/nightlies/">nightlies</a>?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-18T11:25:54Z2016-10-18T11:25:54ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Hey</p>
<p>Yes, I'm using this type of version. Maybe it's related to
Angular somehow hm..?</p>
<p>Is it possible to wait with tests by few seconds after logging
process to the moment when page will be fully loaded(Angular may
check some privileges/permissions on beginning before will display
proper elements on page)?</p>
<p>Generated HTML report in sitemap section should list all scanned
URLs or only part?</p>
<p>Arachni supports below type of URL?<br>
<a href="https://example.com/module#section/element">https://example.com/module#section/element</a></p>
<p>Thanks for quick response</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-18T11:27:52Z2016-10-18T11:27:52ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Check out this option: <a href="https://github.com/Arachni/arachni/wiki/Command-line-user-interface#browser-cluster-wait-for-element">
https://github.com/Arachni/arachni/wiki/Command-line-user-interface...</a></p>
<p>Arachni supports URLs with fragments but only internally, as
part of the crawl.<br>
If you set them via options the fragment will be removed.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-18T12:56:55Z2016-10-18T12:56:57ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>By reading this thread, I believe that i have a similar problem
with the new thread i opened yesterday. <a href="http://support.arachni-scanner.com/discussions/questions/13047-create-logon-script-for-angularjs">
http://support.arachni-scanner.com/discussions/questions/13047-crea...</a></p>
<p>Owasp Juice Shop uses the same URL structure. Perhaps you can
try using it to sort this out.</p></div>marcio.rochatag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T08:04:56Z2016-10-20T08:04:56ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Hello Tasos :)</p>
<p>I tested latest nightly version published yesterday by using
proposed parameter and it looks Arachni was able to find pages(I
used trainer as first step) in our application! Big congratulations
for You :)<br>
I'm going to check it context of specific checks.<br>
By the way my apologies for my unfair opinion in topic created by
Marcio.<br>
I will let You know soon about results in real conditions.</p>
<p>Have a great day!</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T09:51:58Z2016-10-20T09:53:15ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>First scan found all pages, but each next wasn't able to repeat
it.<br>
Also with next scans I noticed those info:</p>
<ol>
<li>
<p>[-] [framework/parts/audit#audit_page:89] [HTTP: 200] <a href="https://example.com/web/libs.js">https://example.com/web/libs.js</a>
[-] [framework/parts/audit#audit_page:90] [operation_timedout]
Timeout was reached [~] Analysis resulted in 0 usable paths. [~]
DOM depth: 0 (Limit: 5)</p>
<p>[*] Workload exhausted, waiting for new pages from the
browser-cluster... [~] BrowserCluster: Pending jobs: 1 [~] Worker:
Retrying (3/6) due to time out: # time= timed_out=false></p>
</li>
<li>
<p>[-] Worker: Job timed-out 6 times: # time=10.239967
timed_out=true></p>
</li>
</ol>
<p>First scan found all pages and was crawling by 40minutes, next
took around 2minutes hm..</p>
<p>Is it related somehow to:<br>
--scope-auto-redundant ?</p>
<p>Also resolved newly spotted error messages related to:<br>
--http-response-max-size</p>
<p>and inf about open files (I used ulimit -n xxxx option)</p>
<p>Any thoughts? All was fine and next scans works only by 2
minutes hm..</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T09:54:07Z2016-10-20T09:54:07ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>You need to increase the <code>--http-request-timeout</code>
option.<br>
Also, how many scans are you running at the same time?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T10:05:24Z2016-10-20T10:06:16ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Arachni finished processing in 00:01:14</p>
<p>One scan. Do You mean --browser-cluster-pool-size parameter?<br>
Example command:</p>
<p>./bin/arachni <a href="https://example.com/web/#/testRuns">https://example.com/web/#/testRuns</a>
--report-save-path=trainer.afr
--plugin=login_script:script=login.rb --scope-include-subdomains
--checks trainer --scope-exclude-pattern="logout|assets|login"
--browser-cluster-wait-for-element='testRuns:#my-test-runs-nav'
--browser-cluster-wait-for-element='testRuns:#mainView'
http-request-timeout=30000</p>
<p>Also I was trying to check this parameter:<br>
--scope-auto-redundant= For 10 and 100 value, Arachni found one
more page with highest value</p>
<p>And mentioned:<br>
--http-response-max-size=1000000</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T10:08:00Z2016-10-20T10:08:00ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Like I said, hashes are not supported for URLs provided via
options and even if they were you're formating your command the
wrong way.<br>
What is actually being run by the command you mentioned is
<code>./bin/arachni https://example.com/web/</code> because
<code>#</code> starts comments in bash.</p>
<p>Instead, use this:</p>
<pre>
<code>./bin/arachni https://example.com/web/ --report-save-path=trainer.afr --plugin=login_script:script=login.rb --scope-include-subdomains --checks trainer --scope-exclude-pattern="logout|assets|login" --browser-cluster-wait-for-element='testRuns:#my-test-runs-nav' --browser-cluster-wait-for-element='testRuns:#mainView' http-request-timeout=30000</code>
</pre></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T10:15:32Z2016-10-20T10:15:32ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>The main problem here is that when I'm typing /web/ I'm
redirected to /web/#/testRuns as it's default page.<br>
Even with this URL Arachni is finishing processing in: 00:02:49<br>
Is it possible to encode this # or use decoded version of it? For
example %23</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T10:18:45Z2016-10-20T10:18:45ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Like I've already mentioned multiple times, you cannot pass URLs
with fragments via the options.<br>
The system will load <code>/web/</code> and pass it to the
browsers, which will then load <code>/web/</code> and get
redirected to <code>/web/#/testRuns</code> and grab the relevant
page.</p>
<p>If you still have issues with the scan it must be something
else, let me know how it goes.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T13:04:32Z2016-10-20T13:04:32ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>I'm struggling with it by entire day. Few hours ago for the same
parameters sitemap included 30 pages, but now only 9 pages. I tried
different parameters for --browser-cluster-wait-for-element but in
results I found 9 pages for different cases.<br>
btw. I used this parameter and I tried to use it twice as it's
multiple in use but in generated report in html form there is info
only about one.</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T13:06:10Z2016-10-20T13:06:10ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Where there any errors?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T13:12:41Z2016-10-20T13:13:24ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>No, I can see only repeatable messages like this:</p>
<pre>
<code>Harvesting HTTP responses...
[12:36:43][Step 3/5] [~] Depending on server responsiveness and network conditions this may take a while.
[12:36:43][Step 3/5]
[12:36:43][Step 3/5] [*] Workload exhausted, waiting for new pages from the browser-cluster...
[12:36:43][Step 3/5] [~] BrowserCluster: Pending jobs: 129
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 128
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 127
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 126
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 125
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 174
[12:36:44][Step 3/5] [~] BrowserCluster: Pending jobs: 258
[12:36:45][Step 3/5] [~] Worker: Waiting for "#mainView" to appear for: https://example.com/web/#/testRuns
[12:36:45][Step 3/5] [~] Worker: "#mainView" appeared for: https://example.com/web/#/testRuns</code>
</pre></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T13:17:41Z2016-10-20T13:17:41ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>That's normal, seems to be working fine.<br>
Unfortunately there's no way to no what's going on without access
to the web application.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T17:19:08Z2016-10-20T17:19:08ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Two questions:<br>
1) I wonder if my session is still active because probably
Arachni(or something else affected the process) stopped processing
2h 30minutes ago(last log) on steps:</p>
<pre>
<code>[14:54:01][Step 3/5] [~] Worker: Waiting for "#user-nav" to appear for: https://examplea.com/web/
[14:54:11][Step 3/5] [-] Worker: "#user-nav" did not appear for: https://example.com/web/
[14:54:29][Step 3/5] [~] BrowserCluster: Pending jobs: 948
[14:54:34][Step 3/5] [~] Worker: Waiting for "#user-nav" to appear for: https://example.com/web/
[14:54:44][Step 3/5] [-] Worker: "#user-nav" did not appear for: https://example.com/web/
[14:55:02][Step 3/5] [~] BrowserCluster: Pending jobs: 947</code>
</pre>
<p>Is it possible to include additional verification to check if
session is still active? hm..</p>
<p>2) Is it possible to save part of report in this situation? I
wonder about found issues, but mainly about sitemap. I used only
general timeout set to 12hours.</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-20T17:36:15Z2016-10-20T17:36:15ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>1) No, either the session is valid or it's not, I don't think
there isn't anything else to check.<br>
2) Hit "Enter", that'll present a control screen, hit "g" and
"Enter" to generate a report during runtime.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:08:47Z2016-10-21T07:08:47ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Hey Tasos</p>
<p>Process stopped on 'Pending jobs: 1439' after 5 hours of
processing:<br></p>
<pre>
<code>[05:12:03][Step 4/6] [*] Path Traversal: Analyzing response #863458 for header input 'extra_arachni_input' pointing to: 'https://example.com/assets/js/example.js'
[05:12:04][Step 4/6]
[05:12:04][Step 4/6] [*] [HTTP: 200] https://example.com/assets/js/Modernizr/modernizr.custom.75913.js
[05:12:04][Step 4/6] [~] Analysis resulted in 0 usable paths.
[05:12:04][Step 4/6] [~] DOM depth: 0 (Limit: 10)
[05:12:04][Step 4/6] [*] Mixed Resource: Checking...
[05:12:04][Step 4/6] [*] Harvesting HTTP responses...
[05:12:04][Step 4/6] [~] Depending on server responsiveness and network conditions this may take a while.
[05:12:04][Step 4/6] [*] Harvesting HTTP responses...
[05:12:04][Step 4/6] [~] Depending on server responsiveness and network conditions this may take a while.
[05:12:04][Step 4/6]
[05:12:04][Step 4/6] [*] Workload exhausted, waiting for new pages from the browser-cluster...
[05:12:04][Step 4/6] [~] BrowserCluster: Pending jobs: 1439</code>
</pre>
<p>Also stopped few times before, for example in this
place:<br></p>
<pre>
<code>[20:52:56][Step 3/5] [*] Blind NoSQL Injection (differential analysis): Got control verification response for cookie variable 'PLAY_FLASH' with action 'https://example.com/assets/js/example.timezones.js'.
[20:52:56][Step 3/5]
[20:52:56][Step 3/5] [*] Workload exhausted, waiting for new pages from the browser-cluster...
[20:52:56][Step 3/5] [~] BrowserCluster: Pending jobs: 1094
[20:52:58][Step 3/5] [~] BrowserCluster: Pending jobs: 1093
[20:53:04][Step 3/5] [~] BrowserCluster: Pending jobs: 1092
[20:53:07][Step 3/5] [~] BrowserCluster: Pending jobs: 1091
[20:53:25][Step 3/5] [~] BrowserCluster: Pending jobs: 1090
[20:53:28][Step 3/5] [~] BrowserCluster: Pending jobs: 1089
[20:53:42][Step 3/5] [~] BrowserCluster: Pending jobs: 1088
[20:53:50][Step 3/5] [~] BrowserCluster: Pending jobs: 1087
[20:53:59][Step 3/5] [~] BrowserCluster: Pending jobs: 1086
[20:54:12][Step 3/5] [~] BrowserCluster: Pending jobs: 1085
[20:54:16][Step 3/5] [~] BrowserCluster: Pending jobs: 1084
[20:54:30][Step 3/5] [~] BrowserCluster: Pending jobs: 1083
[20:54:33][Step 3/5] [~] BrowserCluster: Pending jobs: 1082
[20:54:34][Step 3/5] [~] BrowserCluster: Pending jobs: 1081
[20:54:51][Step 3/5] [~] BrowserCluster: Pending jobs: 1080
[20:54:51][Step 3/5] [~] BrowserCluster: Pending jobs: 1079
[20:54:56][Step 3/5] [~] BrowserCluster: Pending jobs: 1078
[20:55:09][Step 3/5] [~] BrowserCluster: Pending jobs: 1077
[20:55:16][Step 3/5] [~] BrowserCluster: Pending jobs: 1076
[20:55:26][Step 3/5] [~] BrowserCluster: Pending jobs: 1075
[20:55:39][Step 3/5] [~] BrowserCluster: Pending jobs: 1074
[20:55:48][Step 3/5] [~] BrowserCluster: Pending jobs: 1073
[20:56:02][Step 3/5] [~] BrowserCluster: Pending jobs: 1072
[20:56:10][Step 3/5] [~] BrowserCluster: Pending jobs: 1071
[20:56:24][Step 3/5] [~] BrowserCluster: Pending jobs: 1070
[20:56:32][Step 3/5] [~] BrowserCluster: Pending jobs: 1069
[20:56:46][Step 3/5] [~] BrowserCluster: Pending jobs: 1068
[20:56:54][Step 3/5] [~] BrowserCluster: Pending jobs: 1067
[20:57:06][Step 3/5] [~] BrowserCluster: Pending jobs: 1066
[20:57:16][Step 3/5] [~] BrowserCluster: Pending jobs: 1065
[20:57:28][Step 3/5] [~] BrowserCluster: Pending jobs: 1064
[20:57:39][Step 3/5] [~] BrowserCluster: Pending jobs: 1063
[20:57:43][Step 3/5] [~] BrowserCluster: Pending jobs: 1062
[20:57:47][Step 3/5] [~] BrowserCluster: Pending jobs: 1061
[20:57:50][Step 3/5] [~] BrowserCluster: Pending jobs: 1060
[20:58:04][Step 3/5] [~] BrowserCluster: Pending jobs: 1059
[20:58:07][Step 3/5] [~] BrowserCluster: Pending jobs: 1058
[20:58:26][Step 3/5] [~] BrowserCluster: Pending jobs: 1057
[20:58:28][Step 3/5] [~] BrowserCluster: Pending jobs: 1056
[20:58:44][Step 3/5] [~] BrowserCluster: Pending jobs: 1055
[20:58:51][Step 3/5] [~] BrowserCluster: Pending jobs: 1054
[20:58:55][Step 3/5] [~] BrowserCluster: Pending jobs: 1053
[20:59:05][Step 3/5] [~] BrowserCluster: Pending jobs: 1052
[20:59:15][Step 3/5] [~] BrowserCluster: Pending jobs: 1051
[20:59:26][Step 3/5] [~] BrowserCluster: Pending jobs: 1050
[20:59:30][Step 3/5] [~] BrowserCluster: Pending jobs: 1049
[20:59:36][Step 3/5] [~] BrowserCluster: Pending jobs: 1048
[20:59:50][Step 3/5] [~] BrowserCluster: Pending jobs: 1047
[20:59:58][Step 3/5] [~] BrowserCluster: Pending jobs: 1046
[21:00:12][Step 3/5] [~] BrowserCluster: Pending jobs: 1045
[21:00:20][Step 3/5] [~] BrowserCluster: Pending jobs: 1044
[21:00:32][Step 3/5] [~] BrowserCluster: Pending jobs: 1043
[21:00:43][Step 3/5] [~] BrowserCluster: Pending jobs: 1042
[21:00:49][Step 3/5] [~] BrowserCluster: Pending jobs: 1041
[21:01:04][Step 3/5] [~] BrowserCluster: Pending jobs: 1040
[21:01:09][Step 3/5] [~] BrowserCluster: Pending jobs: 1039
[21:01:27][Step 3/5] [~] BrowserCluster: Pending jobs: 1038
[21:01:30][Step 3/5] [~] BrowserCluster: Pending jobs: 1037
[21:01:49][Step 3/5] [~] BrowserCluster: Pending jobs: 1036
[21:01:53][Step 3/5] [~] BrowserCluster: Pending jobs: 1035
[21:02:12][Step 3/5] [~] BrowserCluster: Pending jobs: 1034
[21:02:15][Step 3/5] [~] BrowserCluster: Pending jobs: 1033
[21:02:17][Step 3/5] [~] BrowserCluster: Pending jobs: 1032
[21:02:20][Step 3/5] [~] BrowserCluster: Pending jobs: 1031
[21:02:40][Step 3/5] [~] BrowserCluster: Pending jobs: 1030
[21:02:42][Step 3/5] [~] BrowserCluster: Pending jobs: 1029
[21:03:02][Step 3/5] [~] BrowserCluster: Pending jobs: 1028
[21:03:24][Step 3/5] [~] BrowserCluster: Pending jobs: 1027
[21:03:46][Step 3/5] [~] BrowserCluster: Pending jobs: 1026
[21:03:50][Step 3/5] [~] BrowserCluster: Pending jobs: 1025
[21:04:07][Step 3/5] [~] BrowserCluster: Pending jobs: 1024
[21:04:29][Step 3/5] [~] BrowserCluster: Pending jobs: 1023
[21:04:50][Step 3/5] [~] BrowserCluster: Pending jobs: 1022
[21:05:07][Step 3/5] [~] BrowserCluster: Pending jobs: 1021
[21:05:11][Step 3/5] [~] BrowserCluster: Pending jobs: 1020
[21:05:34][Step 3/5] [~] BrowserCluster: Pending jobs: 1019
[21:05:55][Step 3/5] [~] BrowserCluster: Pending jobs: 1018
[21:06:17][Step 3/5] [~] BrowserCluster: Pending jobs: 1017
[21:06:39][Step 3/5] [~] BrowserCluster: Pending jobs: 1016
[21:07:01][Step 3/5] [~] BrowserCluster: Pending jobs: 1015
[21:07:05][Step 3/5] [~] BrowserCluster: Pending jobs: 1014
[21:07:28][Step 3/5] [~] BrowserCluster: Pending jobs: 1013</code>
</pre>
<ol>
<li>
<p>Maybe memory issue in context of combination of all checks with
all assets to scan? I was trying to decrease amount of workers and
re-spawn parameter. A little bit better but still frozen processing
hm..</p>
</li>
<li>
<p>Is it possible to divide checks to smaller(logical) groups? I
wonder how can I split checks to ensure Arachni will be able to
find all bugs? There is some relation between checks like
combo?</p>
</li>
</ol>
<p>Thanks in advance</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:10:29Z2016-10-21T07:10:29ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Can you show me your configuration please?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:31:07Z2016-10-21T07:34:25ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>I tried many configurations. I was using default parameters but
also different values.<br>
Maybe I'm doing something wrong hm..<br>
Examples:</p>
<pre>
<code>bin/arachni
https://example.com/web/
--report-save-path=report.afr
--plugin=login_script:script=logindata.rb
--scope-include-subdomains
--plugin=metrics
--scope-exclude-pattern="logout"
--browser-cluster-wait-for-element='testRuns:.navbar-brand'
--browser-cluster-pool-size=3
--http-response-max-size=1000000
--http-request-timeout=60000
--scope-auto-redundant=100
--scope-dom-depth-limit=10
--browser-cluster-worker-time-to-live=50
--timeout=12:00:00
--audit-links
--audit-forms
--audit-headers
--audit-jsons
--audit-xmls
--audit-ui-inputs
--audit-ui-forms
--audit-parameter-names
--audit-with-raw-payloads
--audit-with-extra-parameter
--audit-with-both-methods
--input-force</code>
</pre>
<p>(In this above I was trying to exclude Cookies as I was thinking
maybe it was affected somehow by tests. That is the reason why I
listed audits separately)</p>
<pre>
<code>bin/arachni
https://example.com/web/
--report-save-path=report.afr
--plugin=login_script:script=logindata.rb
--scope-include-subdomains
--plugin=metrics
--scope-exclude-pattern="logout"
--browser-cluster-wait-for-element='testRuns:.navbar-brand'
--browser-cluster-pool-size=6
--http-response-max-size=1000000
--http-request-timeout=60000
--scope-auto-redundant=100
--scope-dom-depth-limit=5
--browser-cluster-worker-time-to-live=50
--timeout=12:00:00
--checks trainer</code>
</pre></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:34:35Z2016-10-21T07:34:35ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Configs look OK.</p>
<p>When you said that the system stopped at the messages you
mentioned, did you mean that it just started showing them or that
it completely stopped showing anything at some point? Like
freezing.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:40:41Z2016-10-21T07:40:41ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Yes Sir, exactly, completely stopped showing anything, it will
be frozen probably to the moment when Arachni will invoke timeout
set to in this example to 12hours.</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T07:44:36Z2016-10-21T07:44:36ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Any chance I can be given access to the webapp to try and
reproduce the issue?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T08:09:04Z2016-10-21T08:13:32ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>I was thinking about it, but unfortunately it's not possible,
policy.</p>
<p>Maybe another type of workaround with decreasing scope to
smaller chunks ? Is it possible to split checks to smaller groups
without impact on scan results? hm.. btw after 5hours logs had
300MB..</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T09:29:36Z2016-10-21T09:29:36ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>I was trying to compare two approaches:</p>
<p>--checks=*xss* vs<br>
--checks=trainer,*xss*</p>
<p>First finished processing quickly(still not able to find lots of
pages from menu hm..).<br>
Second stopped(frozen):<br></p>
<pre>
<code>[09:14:23][Step 4/6] [~] BrowserCluster: Pending jobs: 886
[09:14:29][Step 4/6] [~] BrowserCluster: Pending jobs: 885
[09:14:44][Step 4/6] [~] BrowserCluster: Pending jobs: 884
[09:14:50][Step 4/6] [~] BrowserCluster: Pending jobs: 883
[09:15:06][Step 4/6] [~] BrowserCluster: Pending jobs: 882
[09:15:13][Step 4/6] [~] BrowserCluster: Pending jobs: 881
[09:15:28][Step 4/6] [~] BrowserCluster: Pending jobs: 880
[09:15:35][Step 4/6] [~] BrowserCluster: Pending jobs: 879
[09:15:50][Step 4/6] [~] BrowserCluster: Pending jobs: 878
[09:15:56][Step 4/6] [~] BrowserCluster: Pending jobs: 877
[09:16:00][Step 4/6] [~] BrowserCluster: Pending jobs: 876
[09:16:12][Step 4/6] [~] BrowserCluster: Pending jobs: 875
[09:16:21][Step 4/6] [~] BrowserCluster: Pending jobs: 874
[09:16:28][Step 4/6] [~] BrowserCluster: Pending jobs: 873
[09:16:37][Step 4/6] [~] BrowserCluster: Pending jobs: 872
[09:17:00][Step 4/6] [~] BrowserCluster: Pending jobs: 871
[09:17:22][Step 4/6] [~] BrowserCluster: Pending jobs: 870
[09:17:45][Step 4/6] [~] BrowserCluster: Pending jobs: 869
[09:18:02][Step 4/6] [~] BrowserCluster: Pending jobs: 868
[09:18:24][Step 4/6] [~] BrowserCluster: Pending jobs: 867
[09:18:28][Step 4/6] [~] BrowserCluster: Pending jobs: 866
[09:18:50][Step 4/6] [~] BrowserCluster: Pending jobs: 865
[09:19:12][Step 4/6] [~] BrowserCluster: Pending jobs: 864
[09:19:16][Step 4/6] [~] BrowserCluster: Pending jobs: 863
[09:19:38][Step 4/6] [~] BrowserCluster: Pending jobs: 862
[09:19:58][Step 4/6] [~] BrowserCluster: Pending jobs: 861
[09:20:20][Step 4/6] [~] BrowserCluster: Pending jobs: 860
[09:20:42][Step 4/6] [~] BrowserCluster: Pending jobs: 859</code>
</pre>
<p>In context of freezing maybe it's related to trainer module?<br>
Maybe I will try to exclude trainer from checks if it's
possible:<br>
--checks=*,-trainer I was analyzing CPU and Disk during Arachni
scan with trainer and.. it seems in last minutes/phase CPU had
usage on 0% level.<br>
Maybe it will be somehow useful info.</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T09:37:04Z2016-10-21T09:37:04ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>The trainer can find more pages, so maybe one of those creates
the problem.<br>
You can also try <code>--output-debug=3</code>, that'll print a
large amount of info which could help.</p>
<p>About changing the scope, I don't know if that'll help, you can
try the <a href="https://github.com/Arachni/arachni/wiki/Command-line-user-interface#scope">
scope options</a>.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T13:46:02Z2016-10-21T13:46:02ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>Good idea with debug mode, I need to check it more
carefully.</p>
<p>I tried to scan page by using --checks=trainer,*xss* parameter.
All looks good but later it stopped processing again hm..<br></p>
<pre>
<code>[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#capture:1757] Worker: Ignoring, already seen.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.1] [!!] [browser#response_handler:1565] Worker: Got response: https://example/assets/img/Example-logo-mono.svg
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1582] Worker: Injected custom JS.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1588] Worker: Asset detected, will not store.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1565] Worker: Got response: https://example/assets/img/login-page-bg.png
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1582] Worker: Injected custom JS.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1588] Worker: Asset detected, will not store.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.3] [!!] [browser#response_handler:1565] Worker: Got response: https://example/assets/font/fontawesome-webfont.woff?v=3.2.1
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1582] Worker: Injected custom JS.
[13:24:40][Step 4/6] [2016-10-21 13:21:43 +0000 - 0.0] [!!] [browser#response_handler:1588] Worker: Asset detected, will not store.
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.1] [!!] [browser#response_handler:1565] Worker: Got response: https://example/assets/img/example-sign-in-brand.svg
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.0] [!!] [browser#response_handler:1582] Worker: Injected custom JS.
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.0] [!!] [browser#response_handler:1588] Worker: Asset detected, will not store.
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.1] [!!] [browser#goto:333] Worker: ...done.
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 6.6] [!!] [browser/javascript#wait_till_ready:165] Waiting for custom JS...
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.0] [!!] [browser/javascript#wait_till_ready:174] ...done.
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.1] [!!] [browser#wait_for_pending_requests:1340] Worker: Waiting for 1 requests to complete:
[13:24:40][Step 4/6] [2016-10-21 13:21:44 +0000 - 0.0] [!!] [browser#wait_for_pending_requests:1345] Worker: * Still reading request data.</code>
</pre>
<p>I will try to start it again with debug set to 3 instead of 2 to
check where it will stop. Maybe issue with parsing request data or
untypical request or something else. I will let You know
later..</p>
<p>Also I'm going to check scope parameters hmm..</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T15:10:48Z2016-10-21T15:10:48ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>That's interesting.</p>
<p>There isn't a time-out for this operation, so long as the
browser sends some request data (but not all) and then keeps the
connection open, the system will wait forever.<br>
The browser doesn't do that though, does the page use
web-sockets?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-21T16:05:12Z2016-10-21T16:05:48ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>There is only mechanism which is sending cycle calls by using
REST API hm.. to check current state of page or to update something
under the hood from that what I know.</p></div>Aladdintag:support.arachni-scanner.com,2012-07-01:Comment/410040412016-10-22T07:54:45Z2016-10-22T07:54:45ZHow to configure Arachni to be able to scan pages which contains navigation based on #/Tab paths<div><p>I see, I'll add a timeout for the operation to play it safe.</p></div>Tasos Laskos