Adverse Effects of http 1.1 logic without 'content-length'

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2
Hi

I’m experiencing a situation where the browser “jams” periodically with 4D v14 in remote mode. What happens is after 4-5 refreshes in fairly quick succession, the request hangs and the browser becomes unresponsive. On cancelling the request (i.e. in Safari, clicking the “x” at the end of the URL bar) it becomes responsive again and the request can be re-initiated, usually successfully.

I traced this to being invoked by the new http 1.1 lines in the Active4D shell request handler code - i.e. the part that checks for the presence of the “content-length” response header and closes the connection if not present. If I remove this block, the request hanging phenomenon disappears.

First of all, I’m wondering is this a feature or a bug ? (The browser hanging). Secondly, should we be adding the content-length header programmatically into all responses (but what about stuff like scripts and bits and pieces of static graphics etc - I won’t be able to cover everything).

Thirdly, can this code block be safely disabled to make the browsing smooth again ?

Finally, another new thing that occurs in 4D v14 all the time is that when it is shut down, 4D displays a message saying:

 “Task http connection handler” and 2 others are still running.
 “To let the application finish its work click ok”
 “To terminate and quit click cancel (You may damage your data)”

Despite this, there are no residual web processes running. Anyone else seen this ?

Any light that can be thrown on these issues - much appreciated !

Regards

Peter

P.S. The latest nightly builds are manifesting the “Pragma Cache” response bug again because Active4D’s shell thinks they’re outside the scope of “bugged versions” for WEB SEND RAW DATA whereas in fact they’re not because 4D haven’t fixed it yet but they do seem to have updated the revision number.

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Schumacher-2
Hi Peter,


there is a bug with truncating HTTP responses in 4D. It's fixed in one of the latest nighthly builds.


Peter
  _____  

From: Peter Jakobsson [mailto:[hidden email]]
To: Active4D Developer Discussion List [mailto:[hidden email]]
Sent: Thu, 16 Oct 2014 02:46:34 +0100
Subject: [Active4d-dev] Adverse Effects of http 1.1 logic without 'content-length'

Hi
 
  I’m experiencing a situation where the browser “jams” periodically with 4D v14 in remote mode. What happens is after 4-5 refreshes in fairly quick succession, the request hangs and the browser becomes unresponsive. On cancelling the request (i.e. in Safari, clicking the “x” at the end of the URL bar) it becomes responsive again and the request can be re-initiated, usually successfully.
 
  I traced this to being invoked by the new http 1.1 lines in the Active4D shell request handler code - i.e. the part that checks for the presence of the “content-length” response header and closes the connection if not present. If I remove this block, the request hanging phenomenon disappears.
 
  First of all, I’m wondering is this a feature or a bug ? (The browser hanging). Secondly, should we be adding the content-length header programmatically into all responses (but what about stuff like scripts and bits and pieces of static graphics etc - I won’t be able to cover everything).
 
  Thirdly, can this code block be safely disabled to make the browsing smooth again ?
 
  Finally, another new thing that occurs in 4D v14 all the time is that when it is shut down, 4D displays a message saying:
 
   “Task http connection handler” and 2 others are still running.
   “To let the application finish its work click ok”
   “To terminate and quit click cancel (You may damage your data)”
 
  Despite this, there are no residual web processes running. Anyone else seen this ?
 
  Any light that can be thrown on these issues - much appreciated !
 
  Regards
 
  Peter
 
  P.S. The latest nightly builds are manifesting the “Pragma Cache” response bug again because Active4D’s shell thinks they’re outside the scope of “bugged versions” for WEB SEND RAW DATA whereas in fact they’re not because 4D haven’t fixed it yet but they do seem to have updated the revision number.
 
  _______________________________________________
  Active4D-dev mailing list
  [hidden email]
  http://list.aparajitaworld.com/listinfo/active4d-dev
  Archives: http://active4d-nabble.aparajitaworld.com/
   
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2

On 16 Oct 2014, at 10:22, Peter Schumacher <[hidden email]> wrote:

> there is a bug with truncating HTTP responses in 4D. It's fixed in one of the latest nighthly builds.

Hi Peter

Thanks for the reply.

I was testing with nightly build 171455 which I thought was the latest.

Is there a new one ?

4D need to support Active4D more and get bugs fixed that affect Active4D with the highest of priorities.

Peter
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2

I think what might be happening is:

[1] - Active4D shell code closes the connection due to missing “content-length” field in response header

[2] - before the web process gets a chance to expire, the network layer has already accepted (at a lower level) another connection but since it’s closed in the http layer it doesn’t get serviced.

[3] - the browser’s left confused. It thinks it connected at the socket level but it’s not getting any response at http level

Thats my theory anyway with my very limited knowledge of low level networking protocols :)

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Schumacher-2
Peter,

do you send the "raw" data? If so, change it back.

Peter
  _____  

From: Peter Jakobsson [mailto:[hidden email]]
To: Active4D Developer Discussion List [mailto:[hidden email]]
Sent: Thu, 16 Oct 2014 10:54:57 +0100
Subject: Re: [Active4d-dev] Adverse Effects of http 1.1 logic without 'content-length'


  I think what might be happening is:
 
  [1] - Active4D shell code closes the connection due to missing “content-length” field in response header
 
  [2] - before the web process gets a chance to expire, the network layer has already accepted (at a lower level) another connection but since it’s closed in the http layer it doesn’t get serviced.
 
  [3] - the browser’s left confused. It thinks it connected at the socket level but it’s not getting any response at http level
 
  Thats my theory anyway with my very limited knowledge of low level networking protocols :)
 
  Peter
 
  _______________________________________________
  Active4D-dev mailing list
  [hidden email]
  http://list.aparajitaworld.com/listinfo/active4d-dev
  Archives: http://active4d-nabble.aparajitaworld.com/
   
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2
On 16 Oct 2014, at 12:28, Peter Schumacher <[hidden email]> wrote:

> do you send the "raw" data? If so, change it back.


Hi Peter

I’ve used both WEB SEND RAW DATA and WEB SEND BLOB. The browser hanging isn’t caused by either of those two commands, it’s caused by the “Connection Close” field in the response header. When I remove that block of code, it stops hanging.

Thats why I wondered if this was a “feature or a bug”.

i.e. am I seeing expected behaviour rather than adverse behaviour in the event that the response header doesn’t contain a “content-length” field.

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
In reply to this post by Peter Schumacher-2
> there is a bug with truncating HTTP responses in 4D. It's fixed in one of the latest nighthly builds.

4D is determined to continually break the web server.

I updated the shell recently, please download the latest and replace A4D_Init. If that doesn't work, then as a temporary workaround insert this line at line 53 of A4D_Init:

<>A4D_CanSendRawData:=False

Regards,

  Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
In reply to this post by Peter Jakobsson-2
> I’ve used both WEB SEND RAW DATA and WEB SEND BLOB. The browser hanging isn’t caused by either of those two commands, it’s caused by the “Connection Close” field in the response header. When I remove that block of code, it stops hanging.
>
> Thats why I wondered if this was a “feature or a bug”.

It's a bug. The 4D web server is supposed to add the Content-Length header. Please capture the http response (using the original shell code), I'd like to see the headers it's generating.

Regards,

  Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2

On 16 Oct 2014, at 13:52, Aparajita Fishman <[hidden email]> wrote:

> It's a bug. The 4D web server is supposed to add the Content-Length header. Please capture the http response (using the original shell code), I'd like to see the headers it's generating.

ok - will do !

Thanks for the enlightenment. Things are starting to make sense now.

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2
In reply to this post by aparajita

On 16 Oct 2014, at 13:52, Aparajita Fishman <[hidden email]> wrote:

> It's a bug. The 4D web server is supposed to add the Content-Length header.

Aparajita -

Sorry if I’m missing something but isn’t the shell testing for the presence of the “content-header” http response field too far upstream ?

At the point where the shell tests for “content-header” the response header has been wholly generated by Active4D and 4D hasn’t yet had a look in. It isn’t until the execution of either WEB SEND RAW DATA or WEB SET HTTP HEADER that 4D could possibly add anything and by that time it’s too late because the shell has already added the close-connection command having not detected any content header field.

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects... - correction

Peter Jakobsson-2
In reply to this post by aparajita

Sorry - in my last message I of course meant “content-length” not “content-header”.

Apologies for the confusion.

Peter


On 16 Oct 2014, at 15:17, Peter Jakobsson <[hidden email]> wrote:

> where the shell tests for “content-header” the response…


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
In reply to this post by Peter Jakobsson-2
> At the point where the shell tests for “content-header” the response header has been wholly generated by Active4D and 4D hasn’t yet had a look in.

Actually I was wrong, Active4D generates the Content-Length header.

Regards,

  Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
In reply to this post by Peter Jakobsson-2
> At the point where the shell tests for “content-header” the response header has been wholly generated by Active4D and 4D hasn’t yet had a look in.

I'm still waiting to see a capture of the http response so I can see what 4D is doing with the headers.

Regards,

 Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2

On 17 Oct 2014, at 13:47, Aparajita Fishman <[hidden email]> wrote:

> I'm still waiting to see a capture of the http response so I can see what 4D is doing with the headers.

Apologies.

I sent them last night but it went to the list by mistake and the list server must have junked it since it had images attached.

Now sent by PM.

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
In reply to this post by Peter Jakobsson-2
> I’ve used both WEB SEND RAW DATA and WEB SEND BLOB. The browser hanging isn’t caused by either of those two commands, it’s caused by the “Connection Close” field in the response header. When I remove that block of code, it stops hanging.

Removing that block of code turns out to be the right thing.

No Content-Length header was generated when the status was 304 (Not Modified), which falsely triggered the adding of "Connection: close".

In any case, I looked at the way real web servers (apache, nginx) handle 304 responses, and they do not use "Connection: close". So lines 84-96 of A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec says..." along with the entire following If block) should be removed entirely.

Regards,

 Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Peter Jakobsson-2

On 19 Oct 2014, at 02:17, Aparajita Fishman <[hidden email]> wrote:

> In any case, I looked at the way real web servers (apache, nginx) handle 304 responses, and they do not use "Connection: close". So lines 84-96 of A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec says..." along with the entire following If block) should be removed entirely.

Great !

Many thanks for looking into that.

Peter

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Perkins, Bradley D
In reply to this post by aparajita
We're getting ready to upgrade a V11-based system to v14.2 with Active4D
6.1. We use NTK for the web server.

I'm occasionally seeing "Connection Reset" errors when serving Active4D
pages. I have not followed this thread closely, but would your
recommendation "Removing that block of code turns out to be the right
thing" help? If so, what/where is that "block of code"?

Thanks,

Brad Perkins

On 10/18/14 6:17 PM, "Aparajita Fishman" <[hidden email]>
wrote:

>> I¹ve used both WEB SEND RAW DATA and WEB SEND BLOB. The browser hanging
>>isn¹t caused by either of those two commands, it¹s caused by the
>>³Connection Close² field in the response header. When I remove that
>>block of code, it stops hanging.
>
>Removing that block of code turns out to be the right thing.
>
>No Content-Length header was generated when the status was 304 (Not
>Modified), which falsely triggered the adding of "Connection: close".
>
>In any case, I looked at the way real web servers (apache, nginx) handle
>304 responses, and they do not use "Connection: close". So lines 84-96 of
>A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec says..."
>along with the entire following If block) should be removed entirely.
>
>Regards,
>
> Aparajita
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/
>


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

aparajita
Administrator
> what/where is that "block of code"?

Look at the text below a little more carefully, I said exactly what method and which lines.


>> Removing that block of code turns out to be the right thing.
>>
>> No Content-Length header was generated when the status was 304 (Not
>> Modified), which falsely triggered the adding of "Connection: close".
>>
>> In any case, I looked at the way real web servers (apache, nginx) handle
>> 304 responses, and they do not use "Connection: close". So lines 84-96 of
>> A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec says..."
>> along with the entire following If block) should be removed entirely.

Regards,

  Aparajita

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Perkins, Bradley D
Thanks. I see it now.

-- Brad


On 10/24/14 9:09 AM, "Aparajita Fishman" <[hidden email]>
wrote:

>> what/where is that "block of code"?
>
>Look at the text below a little more carefully, I said exactly what
>method and which lines.
>
>
>>> Removing that block of code turns out to be the right thing.
>>>
>>> No Content-Length header was generated when the status was 304 (Not
>>> Modified), which falsely triggered the adding of "Connection: close".
>>>
>>> In any case, I looked at the way real web servers (apache, nginx)
>>>handle
>>> 304 responses, and they do not use "Connection: close". So lines 84-96
>>>of
>>> A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec
>>>says..."
>>> along with the entire following If block) should be removed entirely.
>
>Regards,
>
>  Aparajita
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/
>


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Adverse Effects of http 1.1 logic without 'content-length'

Perkins, Bradley D
This only appears to affect users using 4D's web server.
I don't see the method "A4D_OnWebConnection" in our structure. We use NTK.

Thanks,

-- Brad

On 10/24/14 11:05 AM, "Perkins, Bradley D" <[hidden email]> wrote:

>Thanks. I see it now.
>
>-- Brad
>
>
>On 10/24/14 9:09 AM, "Aparajita Fishman" <[hidden email]>
>wrote:
>
>>> what/where is that "block of code"?
>>
>>Look at the text below a little more carefully, I said exactly what
>>method and which lines.
>>
>>
>>>> Removing that block of code turns out to be the right thing.
>>>>
>>>> No Content-Length header was generated when the status was 304 (Not
>>>> Modified), which falsely triggered the adding of "Connection: close".
>>>>
>>>> In any case, I looked at the way real web servers (apache, nginx)
>>>>handle
>>>> 304 responses, and they do not use "Connection: close". So lines 84-96
>>>>of
>>>> A4D_OnWebConnection (starting with the comment "//HTTP 1.1 spec
>>>>says..."
>>>> along with the entire following If block) should be removed entirely.
>>
>>Regards,
>>
>>  Aparajita
>>
>>_______________________________________________
>>Active4D-dev mailing list
>>[hidden email]
>>http://list.aparajitaworld.com/listinfo/active4d-dev
>>Archives: http://active4d-nabble.aparajitaworld.com/
>>
>
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/
>


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
12