tag:support.arachni-scanner.com,2012-07-01:/discussions/problems/3914-blind-sqli-timing-checks-and-dosArachni: Discussion 2015-12-07T21:48:59Ztag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-02T22:56:32Z2015-12-07T17:18:31ZBlind SQLi Timing Checks and DoS<div><p>I have been testing Arachni's experimental branch against a few
vulnerable applications (Hackazon in this case) and have run into
an issue where Arachni causes a DoS against a local target web
server using the following options:<br></p>
<pre>
<code>./arachni http://172.x.y.z:8000/ --plugin=autologin:url=http://172.x.y.z:8000/user/login,parameters='username=user&password=test123',check='Logout' --scope-exclude-pattern logout --timeout 02:00:00 --http-request-concurrency=1 --http-request-timeout=10000 --browser-cluster-ignore-images --browser-cluster-pool-size=1 --audit-with-raw-payloads --checks=sql_injection_timing</code>
</pre>
I was able to trace this back to the Blind SQLi timing checks by
monitoring the scan and saw the number of Apache pids and an
excessive number of CLOSE_WAIT's for connections on the target host
when Arachni started the Blind SQLi timing checks. I also monitored
the mysql database on the target for long running queries and the
Apache logs so I could see what Arachni was sending to hang the
server:<br>
<pre>
<code>while a==a; do echo "Apache PIDs: <code>ps -ef | grep apache | grep -v grep | wc -l</code>"; echo "CLOSE_WAIT Connections: <code>netstat -an | grep CLOSE_WAIT | grep &quot;:8000&quot; | wc -l</code>"; mysql -uhackazon -p****** -e"SELECT TIME,INFO FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='hackazon'";echo "Apache Log: <code>grep &quot;sleep(4)&quot; /var/log/apache2/access.log | tail -1 | cut -d&quot;]&quot; -f2</code>"; echo "##########################"; sleep 5; done</code>
</pre>
After a few minutes into the Blind SQLi timing checks, the target
host becomes unresponsive. I've tried the previously mentioned
suggestion for <a href="http://support.arachni-scanner.com/discussions/problems/3898-tuning-arachni-for-lowering-requests-per-second">
rate-limiting concurrent connections</a>, however it doesn't seem
have an affect in this scenario. This issue seems to happen when a
successful Blind SQLi attack occurs against the target within a
query's WHERE clause and large tables:<br>
<pre>
<code>Apache PIDs: 152
CLOSE_WAIT Connections: 152
+------+--------------------------------------------------------------------------------------------+
| TIME | INFO |
+------+--------------------------------------------------------------------------------------------+
| 0 | SELECT TIME,INFO FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='hackazon' |
| 182 | SELECT * FROM 'tbl_categories' WHERE 'tbl_categories'.'categoryID' = '49'=sleep(4)='' LIMIT 1 |
+------+--------------------------------------------------------------------------------------------+
Apache Log: "GET /category/view?id=49'=sleep(4)=' HTTP/1.1" 200 5681 "-" "Arachni/v2.0dev"</code>
</pre>
The attack results in a sleep(4) occurring for every row when the
condition is false, so it creates a very long running query (i.e.
~4 mins for the query above with some other queries taking over 14
mins). Since this happens for a number on inputs on this target, I
believe this is why I'm seeing so many Apache PIDs and CLOSE_WAITs
occurring that eventually kills the server. Is it possible to look
for dangerous conditions such as this with the timing checks and
selectively throttle only the timing checks when a connection has
been open longer than some safe upper limit (i.e. sleep(4) +
some_safe_upper_limit) to ensure a target doesn't get DoS'd?
<p>I tried testing this in the latest nightly build instead of
running directly from the experimental branch, but scans seem to be
hanging at "Processing plugins" with the Nov. 23rd nightly
build.</p></div>wspirestag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-02T23:05:04Z2015-12-02T23:05:04ZBlind SQLi Timing Checks and DoS<div><p>I don't think this has to do with HTTP request throttling, it
looks to be solely the result of the SQL injection payload and I'll
fix it ASAP.</p>
<p>Thanks a lot for the information, I'll let you know once a
nightly is available that fixes this issue.</p>
<p>Cheers</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-02T23:23:41Z2015-12-02T23:23:41ZBlind SQLi Timing Checks and DoS<div><p>Come to think about it again, the high DB query runtimes may not
be related to the payload, but by resource exhaustion due to the
many Apache workers.<br>
Especially since the SQL query has a <code>LIMIT</code> clause.</p>
<p>Does the issue occur after uncommenting this line?:<br>
<a href="https://github.com/Arachni/arachni/blob/experimental/lib/arachni/http/request.rb#L403">
https://github.com/Arachni/arachni/blob/experimental/lib/arachni/ht...</a></p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-03T00:17:06Z2015-12-03T00:17:06ZBlind SQLi Timing Checks and DoS<div><p>I did a little more research, a solution to this seems to only
exist for MySQL >= 5.6.</p>
<p>I figured with the <code>LIMIT</code> clause it wouldn't be the
payload, but from what I could find it turns out that things work
exactly the way you described.</p>
<p>The <code>sleep()</code> function call is executed for each row,
and even when injected as a subquery (<code>(select
sleep(4)</code>) it'll still behave the same way for any version
< 5.6.</p>
<p>If you are worried about stressing remote MySQL servers you
better not load the <code>sql_injection_timing</code> check or
specify the <code>mssql,pgsql</code> platforms which will skip
MySQL payloads.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-03T15:55:38Z2015-12-07T17:18:31ZBlind SQLi Timing Checks and DoS<div><p>I tried uncommenting that line you suggested and the results are
similar as before:<br></p>
<pre>
<code>##########################
Apache PIDs: 152
CLOSE_WAIT Connections: 191
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TIME | INFO |
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 0 | SELECT TIME,INFO FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='hackazon' |
| 417 | SELECT * FROM 'tbl_products' LEFT JOIN 'tbl_category_product' ON 'tbl_category_product'.'productID' = 'tbl_products'.'productID' WHERE 'tbl_category_product'.'categoryID' IN (2,3,4,5,6,7,8,9,10,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52) AND 'tbl_products'.'name' LIKE '%1'=sleep(4)='%' AND 'tbl_products'.'Price' >= '0' AND 'tbl_products'.'Price' <= '0' |
+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</code>
</pre>
I believe the resource exhaustion is an indirect result of the long
running query that's created with the attack payload. All of those
Apache procs are hanging around in <code>CLOSE_WAIT</code> until
those long running queries return to allow the connections to fully
close. I don't think there is anything that Arachni can do to force
that connection to fully close when the DB is tying up the Apache
connections in <code>CLOSE_WAIT</code>.
<p>The reason why I was looking at request throttling is because
throttling the rate of the timing test requests once something like
this occurs during a scan may allow the rest of a scan to finish
while waiting for those long running queries to return for the
timing tests. Another way to handle it once something like this is
detected is to just bail out of the remaining mysql timing checks
altogether and report the vuln and potential DoS condition to the
user. I'm not sure how possible something like that would be
though.</p>
<p>So far I haven't noticed where Blind SQLi has been reported in
any of the scans where I ran into this, even though the payload
seems to be executing. This may be due to the excessive time it
takes for the query to return, but I'm not sure if that would cause
a reporting issue.</p></div>wspirestag:support.arachni-scanner.com,2012-07-01:Comment/386081152015-12-03T16:08:21Z2015-12-03T21:11:04ZBlind SQLi Timing Checks and DoS<div><p>The issues aren't being reported because the server has become
unusable so there's no way to verify them.<br>
This is just the first stage of the timing attacks.</p>
<p>As for throttling the timing attacks, that happens during the
verification phases, it's assumed that the server will make it past
the initial candidate gathering phase though.</p>
<p>There usually are server side timeouts for sql queries to
prevent these types of problems and the scanner can only do so
much.</p>
<p>If you're worried about that happening in real world
circumstances the only solution is to not perform sql injections
using timing attacks against MySQL.</p></div>Tasos Laskos