Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Perkins, Bradley D
So keep latest NTK and Active4D Plugins but reinstall the the old shell
code? That would include using the old A4D_NTKConfig lists.
We can try that. It will have to wait a few weeks because of other project
commitments.

Thanks,

Brad Perkins

On 4/25/17, 9:41 AM, "Active4D-dev on behalf of Aparajita Fishman"
<[hidden email] on behalf of
[hidden email]> wrote:

>I meant only to use the old NTK shell code. Nothing else would have to
>change. If your problems persist and I can’t find the solution, I’ll do
>it for free.
>
>All the best,
>
>  - Aparajita
>
>> On Apr 25, 2017, at 7:21 AM, Perkins, Bradley D <[hidden email]>
>>wrote:
>>
>> Not really. I'd have to:
>> (a) Go back to the old server machine and 4D v14
>> (b) Work with institutional IT to get it back on the network (Hostmaster
>> updates, Switch programming). They tightly control all public facing IP
>> addresses.
>> (c) Identify all changed data since we went live and export/import it
>>back
>> into the v14 Database.
>> As noted that ran fine.
>>
>> I never considered running v14 on the Sierra machines. I was told by 4D
>>it
>> wouldn't be stable.
>>
>> If you mean staying with v15.4 and putting in the old shell code and
>>going
>> back to older NTK and Active4D versions, we could try that, but I have
>> other other project commitments that I must focus on for the next few
>> weeks and can't work on that now. We're just going to have to monitor
>>our
>> systems and live with the problems if they persist.
>>
>> One idea I did have was to compile the NTK demo database under v15.4
>> running same NTK and A4D versions and place that on our development
>> system, reasonably exercise it with AB, and see if the Active4D/NTK
>>server
>> becomes unresponsive.
>>
>> Also, I checked and we are still using the old network layer.
>>
>>
>> Thanks,
>>
>> -- Brad
>>
>> On 4/25/17, 7:30 AM, "Active4D-dev on behalf of Aparajita Fishman"
>> <[hidden email] on behalf of
>> [hidden email]> wrote:
>>
>>> Brad,
>>>
>>> Is it possible to roll back to the old NTK server code? I'd like to
>>>know
>>> if that solves the problem.
>>>
>>> All the best,
>>>
>>> - Aparajita
>>>
>>>> On Apr 24, 2017, at 1:53 PM, Perkins, Bradley D <[hidden email]>
>>>> wrote:
>>>>
>>>> Hi John,
>>>>
>>>> We went from v14 to v15. We're also still running 32-bit server. I'm
>>>>not
>>>> positive but I thought the new network layer was only available in
>>>> 64-bit.
>>>> I will definitely check though. Thanks!
>>>>
>>>> Brad
>>>>
>>>> On 4/24/17, 2:31 PM, "Active4D-dev on behalf of Bellos, John"
>>>> <[hidden email] on behalf of
>>>> [hidden email]> wrote:
>>>>
>>>>> I forget what version of 4D you said you came from, but if it was v13
>>>>> or
>>>>> lower, try your setup with the new network layer OFF in 4D server.
>>>>> We've
>>>>> definitely seen some stability issues with the new network layer ON,
>>>>> using v15.4.
>>>>>
>>>>> Note that if you're using Mac 64-Bit, the new network layer is forced
>>>>> on
>>>>> and you can't turn it off. But on Windows 64 bit server, you can turn
>>>>> this setting off.
>>>>>
>>>>> ________________________________________
>>>>> From: Active4D-dev [[hidden email]] on
>>>>>behalf
>>>>> of
>>>>> Aparajita Fishman [[hidden email]]
>>>>> Sent: Monday, April 24, 2017 4:29 PM
>>>>> To: Active4D Developer Discussion List
>>>>> Subject: Re: [Active4d-dev] Active4D Slowdown then Stall (weird
>>>>> A4D_HTTPD_RequestHandler_* behavior)
>>>>>
>>>>> It’s entirely possible there is some flaw in the new server code. The
>>>>> main change I made to A4D_HTTPD_RequestHandler was the correct
>>>>>handling
>>>>> of keep-alive connections. If you find out any more details, please
>>>>> post
>>>>> them here.
>>>>>
>>>>> All the best,
>>>>>
>>>>> - Aparajita
>>>>>
>>>>>> On Apr 24, 2017, at 8:43 AM, Perkins, Bradley D <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Peter and Aparajita,
>>>>>>
>>>>>> Thank you both for your responses. I have been testing our
>>>>>>development
>>>>>> system with a higher MaxThreads this morning. I haven't been able to
>>>>>> exhaust all of the threads. I was able to crash 4D through. I'll
>>>>>>send
>>>>>> Aparajita the crash log.
>>>>>>
>>>>>> Peter, your observations are very similar to what we've seen. Under
>>>>>> v14
>>>>>> --
>>>>>> and even with the broken keep alive feature -- we could run for a
>>>>>>long
>>>>>> time with no problems. Reboots were usually because I had to reboot
>>>>>> the
>>>>>> Windows server to install MS Security Patches.
>>>>>>
>>>>>> Our production system sees a fairly light load and would normally
>>>>>> never
>>>>>> see the loads that I'm placing on it with ApacheBench unless IT is
>>>>>> running
>>>>>> a cyber scan. I'm letting it run as is with the earlier
>>>>>>configuration
>>>>>> to
>>>>>> see if the problem is because we've exhausted the thread pool. We
>>>>>> haven't
>>>>>> allocated an additional thread since I restarted on Saturday. Our
>>>>>> ThreadStack size was already set to 256 (docs show 128 as default).
>>>>>> As Peter notes, It could be that the problem lies elsewhere such as
>>>>>>in
>>>>>> in
>>>>>> 4D itself.
>>>>>>
>>>>>> We upgraded hardware, OS, 4D, NTK, and Active4D. That is a lot of
>>>>>> change.
>>>>>>
>>>>>> -- Brad
>>>>>>
>>>>>> On 4/24/17, 9:09 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>>> <[hidden email] on behalf of
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing this
>>>>>>> to
>>>>>>> 100 by default in the next release, and I am also adding code to
>>>>>>> reclaim
>>>>>>> unused threads after a configurable delay.
>>>>>>>
>>>>>>> All the best,
>>>>>>>
>>>>>>> - Aparajita
>>>>>>>
>>>>>>>> On Apr 24, 2017, at 4:51 AM, Peter Gutbrod <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Brad,
>>>>>>>>
>>>>>>>> I see something similar since I've upgraded 4D-Server to V16 and
>>>>>>>> NTK/A4D
>>>>>>>> to the newest version.
>>>>>>>>
>>>>>>>> Before the server could run für month without any issue. After the
>>>>>>>> upgrade the server slows down within 1-2 days up the point, where
>>>>>>>> each
>>>>>>>> request can take 10 seconds or even more. According to the 4D
>>>>>>>>admin
>>>>>>>> panel there are cpu or network intensive tasks running. Server is
>>>>>>>> almost
>>>>>>>> idle, tons af free memory for 4D. Nothing suspicious in Windows
>>>>>>>> performance monitor. Rebooting 4D and the slow-down is gone -  for
>>>>>>>> the
>>>>>>>> next couple of hours, then it slowly reappears.
>>>>>>>>
>>>>>>>> I realized the slowdown in a JSON API. First I thought it is
>>>>>>>> something
>>>>>>>> within my A4D code, so I've calculated the time my A4D code needs
>>>>>>>>to
>>>>>>>> run. But I found the slowdown happens while receiving the request.
>>>>>>>> Once
>>>>>>>> A4D starts executing the code, everything runs fast, including
>>>>>>>> sending
>>>>>>>> the response.
>>>>>>>>
>>>>>>>> I've asked Rob from the pluggers, whether he had heard about
>>>>>>>> slowdows
>>>>>>>> with the new version of NTK but he wasn't aware of any problems.
>>>>>>>>
>>>>>>>> In a period, where I wasn't able to restart 4D server
>>>>>>>>periodically,
>>>>>>>> slowdown got so extreme, that 4D clients were unable to login due
>>>>>>>>to
>>>>>>>> timeouts. In that state the only way to restart the 4D server was
>>>>>>>> with a
>>>>>>>> force quit. That convinced me, that the culprit is within 4D V16
>>>>>>>>and
>>>>>>>> not
>>>>>>>> NTK or A4D and I should wait for 4D 16.1 and until then restart 4D
>>>>>>>> Server periodically. That is why I never reported the problem
>>>>>>>>here.
>>>>>>>>
>>>>>>>> As you have a similar problem but with 4D15.4 makes me wondering,
>>>>>>>> whether I'm wrong and the lastest versions of NTK and A4D might at
>>>>>>>> least
>>>>>>>> contribute to the problem.
>>>>>>>>
>>>>>>>> Hope you find the answer and share it with me.
>>>>>>>>
>>>>>>>> Greetings
>>>>>>>>
>>>>>>>> Peter Gutbrod
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaw
>>>>>>>>or
>>>>>>>> ld
>>>>>>>>
>>>>>>>>
>>>>>>>>.com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=ww
>>>>>>>>tj
>>>>>>>> VR
>>>>>>>>
>>>>>>>>
>>>>>>>>q8UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5
>>>>>>>>PK
>>>>>>>> YC
>>>>>>>> 6wobVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>> Archives:
>>>>>>>>
>>>>>>>>
>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabb
>>>>>>>>le
>>>>>>>> .a
>>>>>>>>
>>>>>>>>
>>>>>>>>parajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO
>>>>>>>>8P
>>>>>>>> 5M
>>>>>>>>
>>>>>>>>
>>>>>>>>3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVc
>>>>>>>>HP
>>>>>>>> sD
>>>>>>>> ls&s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawo
>>>>>>>rl
>>>>>>> d.
>>>>>>>
>>>>>>>
>>>>>>>com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtj
>>>>>>>VR
>>>>>>> q8
>>>>>>>
>>>>>>>
>>>>>>>UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKY
>>>>>>>C6
>>>>>>> wo
>>>>>>> bVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>> Archives:
>>>>>>>
>>>>>>>
>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabbl
>>>>>>>e.
>>>>>>> ap
>>>>>>>
>>>>>>>
>>>>>>>arajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P
>>>>>>>5M
>>>>>>> 3-
>>>>>>>
>>>>>>>
>>>>>>>rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPs
>>>>>>>Dl
>>>>>>> s&
>>>>>>> s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>>
>>>>>>
>>>>>>
>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawor
>>>>>>ld
>>>>>> .c
>>>>>>
>>>>>>
>>>>>>om_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVR
>>>>>>q8
>>>>>> UQ
>>>>>>
>>>>>>
>>>>>>mO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6w
>>>>>>ob
>>>>>> Vc
>>>>>> HPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>> Archives:
>>>>>>
>>>>>>
>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabble
>>>>>>.a
>>>>>> pa
>>>>>>
>>>>>>
>>>>>>rajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M
>>>>>>3-
>>>>>> rw
>>>>>>
>>>>>>
>>>>>>SBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls
>>>>>>&s
>>>>>> =V
>>>>>> JzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>>
>>>>>
>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaworl
>>>>>d.
>>>>> co
>>>>>
>>>>>
>>>>>m_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8
>>>>>UQ
>>>>> mO
>>>>>
>>>>>
>>>>>8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobV
>>>>>cH
>>>>> Ps
>>>>> Dls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>> Archives:
>>>>>
>>>>>
>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabble.
>>>>>ap
>>>>> ar
>>>>>
>>>>>
>>>>>ajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M3-
>>>>>rw
>>>>> SB
>>>>>
>>>>>
>>>>>mptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls&s=
>>>>>VJ
>>>>> zL
>>>>> TdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>
>>>> _______________________________________________
>>>> Active4D-dev mailing list
>>>> [hidden email]
>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email]
>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Aparajita Fishman
In reply to this post by Peter Gutbrod
If you were hitting the previous limit of 27 for MaxThreads, increasing to 97 will definitely make a difference.

All the best,

  - Aparajita

> On Apr 25, 2017, at 10:04 AM, Peter Gutbrod <[hidden email]> wrote:
>
> I had set from the beginning:
>
> NumThreads=13
> MaxThreads=97
> ThreadStack=256
> CompressResponse=1
> KeepAliveTime=5
>
> Don't think MaxThreads=97 would make a difference.
>
> Any other values to play with?
>
> Rebooted early in the morning and 12 hours later there are still no
> slowdowns, fingers crossed.
>
> Peter
>> Aparajita Fishman <mailto:[hidden email] <mailto:[hidden email]>>
>> 24. April 2017 um 17:09
>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing this to
>> 100 by default in the next release, and I am also adding code to
>> reclaim unused threads after a configurable delay.
>>
>> All the best,
>>
>> - Aparajita
>>
>>
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
> _______________________________________________
> Active4D-dev mailing list
> [hidden email] <mailto:[hidden email]>
> http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev>
> Archives: http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/>
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Aparajita Fishman
In reply to this post by Perkins, Bradley D
Yes, that’s what I propose. In the meantime I’m going to do some long-term stress testing here.

All the best,

  - Aparajita

> On Apr 25, 2017, at 10:23 AM, Perkins, Bradley D <[hidden email]> wrote:
>
> So keep latest NTK and Active4D Plugins but reinstall the the old shell
> code? That would include using the old A4D_NTKConfig lists.
> We can try that. It will have to wait a few weeks because of other project
> commitments.
>
> Thanks,
>
> Brad Perkins
>
> On 4/25/17, 9:41 AM, "Active4D-dev on behalf of Aparajita Fishman"
> <[hidden email] on behalf of
> [hidden email]> wrote:
>
>> I meant only to use the old NTK shell code. Nothing else would have to
>> change. If your problems persist and I can’t find the solution, I’ll do
>> it for free.
>>
>> All the best,
>>
>> - Aparajita
>>
>>> On Apr 25, 2017, at 7:21 AM, Perkins, Bradley D <[hidden email]>
>>> wrote:
>>>
>>> Not really. I'd have to:
>>> (a) Go back to the old server machine and 4D v14
>>> (b) Work with institutional IT to get it back on the network (Hostmaster
>>> updates, Switch programming). They tightly control all public facing IP
>>> addresses.
>>> (c) Identify all changed data since we went live and export/import it
>>> back
>>> into the v14 Database.
>>> As noted that ran fine.
>>>
>>> I never considered running v14 on the Sierra machines. I was told by 4D
>>> it
>>> wouldn't be stable.
>>>
>>> If you mean staying with v15.4 and putting in the old shell code and
>>> going
>>> back to older NTK and Active4D versions, we could try that, but I have
>>> other other project commitments that I must focus on for the next few
>>> weeks and can't work on that now. We're just going to have to monitor
>>> our
>>> systems and live with the problems if they persist.
>>>
>>> One idea I did have was to compile the NTK demo database under v15.4
>>> running same NTK and A4D versions and place that on our development
>>> system, reasonably exercise it with AB, and see if the Active4D/NTK
>>> server
>>> becomes unresponsive.
>>>
>>> Also, I checked and we are still using the old network layer.
>>>
>>>
>>> Thanks,
>>>
>>> -- Brad
>>>
>>> On 4/25/17, 7:30 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>> <[hidden email] on behalf of
>>> [hidden email]> wrote:
>>>
>>>> Brad,
>>>>
>>>> Is it possible to roll back to the old NTK server code? I'd like to
>>>> know
>>>> if that solves the problem.
>>>>
>>>> All the best,
>>>>
>>>> - Aparajita
>>>>
>>>>> On Apr 24, 2017, at 1:53 PM, Perkins, Bradley D <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Hi John,
>>>>>
>>>>> We went from v14 to v15. We're also still running 32-bit server. I'm
>>>>> not
>>>>> positive but I thought the new network layer was only available in
>>>>> 64-bit.
>>>>> I will definitely check though. Thanks!
>>>>>
>>>>> Brad
>>>>>
>>>>> On 4/24/17, 2:31 PM, "Active4D-dev on behalf of Bellos, John"
>>>>> <[hidden email] on behalf of
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> I forget what version of 4D you said you came from, but if it was v13
>>>>>> or
>>>>>> lower, try your setup with the new network layer OFF in 4D server.
>>>>>> We've
>>>>>> definitely seen some stability issues with the new network layer ON,
>>>>>> using v15.4.
>>>>>>
>>>>>> Note that if you're using Mac 64-Bit, the new network layer is forced
>>>>>> on
>>>>>> and you can't turn it off. But on Windows 64 bit server, you can turn
>>>>>> this setting off.
>>>>>>
>>>>>> ________________________________________
>>>>>> From: Active4D-dev [[hidden email]] on
>>>>>> behalf
>>>>>> of
>>>>>> Aparajita Fishman [[hidden email]]
>>>>>> Sent: Monday, April 24, 2017 4:29 PM
>>>>>> To: Active4D Developer Discussion List
>>>>>> Subject: Re: [Active4d-dev] Active4D Slowdown then Stall (weird
>>>>>> A4D_HTTPD_RequestHandler_* behavior)
>>>>>>
>>>>>> It’s entirely possible there is some flaw in the new server code. The
>>>>>> main change I made to A4D_HTTPD_RequestHandler was the correct
>>>>>> handling
>>>>>> of keep-alive connections. If you find out any more details, please
>>>>>> post
>>>>>> them here.
>>>>>>
>>>>>> All the best,
>>>>>>
>>>>>> - Aparajita
>>>>>>
>>>>>>> On Apr 24, 2017, at 8:43 AM, Perkins, Bradley D <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Peter and Aparajita,
>>>>>>>
>>>>>>> Thank you both for your responses. I have been testing our
>>>>>>> development
>>>>>>> system with a higher MaxThreads this morning. I haven't been able to
>>>>>>> exhaust all of the threads. I was able to crash 4D through. I'll
>>>>>>> send
>>>>>>> Aparajita the crash log.
>>>>>>>
>>>>>>> Peter, your observations are very similar to what we've seen. Under
>>>>>>> v14
>>>>>>> --
>>>>>>> and even with the broken keep alive feature -- we could run for a
>>>>>>> long
>>>>>>> time with no problems. Reboots were usually because I had to reboot
>>>>>>> the
>>>>>>> Windows server to install MS Security Patches.
>>>>>>>
>>>>>>> Our production system sees a fairly light load and would normally
>>>>>>> never
>>>>>>> see the loads that I'm placing on it with ApacheBench unless IT is
>>>>>>> running
>>>>>>> a cyber scan. I'm letting it run as is with the earlier
>>>>>>> configuration
>>>>>>> to
>>>>>>> see if the problem is because we've exhausted the thread pool. We
>>>>>>> haven't
>>>>>>> allocated an additional thread since I restarted on Saturday. Our
>>>>>>> ThreadStack size was already set to 256 (docs show 128 as default).
>>>>>>> As Peter notes, It could be that the problem lies elsewhere such as
>>>>>>> in
>>>>>>> in
>>>>>>> 4D itself.
>>>>>>>
>>>>>>> We upgraded hardware, OS, 4D, NTK, and Active4D. That is a lot of
>>>>>>> change.
>>>>>>>
>>>>>>> -- Brad
>>>>>>>
>>>>>>> On 4/24/17, 9:09 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>>>> <[hidden email] on behalf of
>>>>>>> [hidden email]> wrote:
>>>>>>>
>>>>>>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing this
>>>>>>>> to
>>>>>>>> 100 by default in the next release, and I am also adding code to
>>>>>>>> reclaim
>>>>>>>> unused threads after a configurable delay.
>>>>>>>>
>>>>>>>> All the best,
>>>>>>>>
>>>>>>>> - Aparajita
>>>>>>>>
>>>>>>>>> On Apr 24, 2017, at 4:51 AM, Peter Gutbrod <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Brad,
>>>>>>>>>
>>>>>>>>> I see something similar since I've upgraded 4D-Server to V16 and
>>>>>>>>> NTK/A4D
>>>>>>>>> to the newest version.
>>>>>>>>>
>>>>>>>>> Before the server could run für month without any issue. After the
>>>>>>>>> upgrade the server slows down within 1-2 days up the point, where
>>>>>>>>> each
>>>>>>>>> request can take 10 seconds or even more. According to the 4D
>>>>>>>>> admin
>>>>>>>>> panel there are cpu or network intensive tasks running. Server is
>>>>>>>>> almost
>>>>>>>>> idle, tons af free memory for 4D. Nothing suspicious in Windows
>>>>>>>>> performance monitor. Rebooting 4D and the slow-down is gone -  for
>>>>>>>>> the
>>>>>>>>> next couple of hours, then it slowly reappears.
>>>>>>>>>
>>>>>>>>> I realized the slowdown in a JSON API. First I thought it is
>>>>>>>>> something
>>>>>>>>> within my A4D code, so I've calculated the time my A4D code needs
>>>>>>>>> to
>>>>>>>>> run. But I found the slowdown happens while receiving the request.
>>>>>>>>> Once
>>>>>>>>> A4D starts executing the code, everything runs fast, including
>>>>>>>>> sending
>>>>>>>>> the response.
>>>>>>>>>
>>>>>>>>> I've asked Rob from the pluggers, whether he had heard about
>>>>>>>>> slowdows
>>>>>>>>> with the new version of NTK but he wasn't aware of any problems.
>>>>>>>>>
>>>>>>>>> In a period, where I wasn't able to restart 4D server
>>>>>>>>> periodically,
>>>>>>>>> slowdown got so extreme, that 4D clients were unable to login due
>>>>>>>>> to
>>>>>>>>> timeouts. In that state the only way to restart the 4D server was
>>>>>>>>> with a
>>>>>>>>> force quit. That convinced me, that the culprit is within 4D V16
>>>>>>>>> and
>>>>>>>>> not
>>>>>>>>> NTK or A4D and I should wait for 4D 16.1 and until then restart 4D
>>>>>>>>> Server periodically. That is why I never reported the problem
>>>>>>>>> here.
>>>>>>>>>
>>>>>>>>> As you have a similar problem but with 4D15.4 makes me wondering,
>>>>>>>>> whether I'm wrong and the lastest versions of NTK and A4D might at
>>>>>>>>> least
>>>>>>>>> contribute to the problem.
>>>>>>>>>
>>>>>>>>> Hope you find the answer and share it with me.
>>>>>>>>>
>>>>>>>>> Greetings
>>>>>>>>>
>>>>>>>>> Peter Gutbrod
>>>>>>>>> _______________________________________________
>>>>>>>>> Active4D-dev mailing list
>>>>>>>>> [hidden email]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaw
>>>>>>>>> or
>>>>>>>>> ld
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> .com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=ww
>>>>>>>>> tj
>>>>>>>>> VR
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> q8UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5
>>>>>>>>> PK
>>>>>>>>> YC
>>>>>>>>> 6wobVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>> Archives:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabb
>>>>>>>>> le
>>>>>>>>> .a
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> parajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO
>>>>>>>>> 8P
>>>>>>>>> 5M
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVc
>>>>>>>>> HP
>>>>>>>>> sD
>>>>>>>>> ls&s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawo
>>>>>>>> rl
>>>>>>>> d.
>>>>>>>>
>>>>>>>>
>>>>>>>> com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtj
>>>>>>>> VR
>>>>>>>> q8
>>>>>>>>
>>>>>>>>
>>>>>>>> UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKY
>>>>>>>> C6
>>>>>>>> wo
>>>>>>>> bVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>> Archives:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabbl
>>>>>>>> e.
>>>>>>>> ap
>>>>>>>>
>>>>>>>>
>>>>>>>> arajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P
>>>>>>>> 5M
>>>>>>>> 3-
>>>>>>>>
>>>>>>>>
>>>>>>>> rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPs
>>>>>>>> Dl
>>>>>>>> s&
>>>>>>>> s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawor
>>>>>>> ld
>>>>>>> .c
>>>>>>>
>>>>>>>
>>>>>>> om_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVR
>>>>>>> q8
>>>>>>> UQ
>>>>>>>
>>>>>>>
>>>>>>> mO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6w
>>>>>>> ob
>>>>>>> Vc
>>>>>>> HPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>> Archives:
>>>>>>>
>>>>>>>
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabble
>>>>>>> .a
>>>>>>> pa
>>>>>>>
>>>>>>>
>>>>>>> rajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M
>>>>>>> 3-
>>>>>>> rw
>>>>>>>
>>>>>>>
>>>>>>> SBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls
>>>>>>> &s
>>>>>>> =V
>>>>>>> JzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>>
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaworl
>>>>>> d.
>>>>>> co
>>>>>>
>>>>>>
>>>>>> m_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8
>>>>>> UQ
>>>>>> mO
>>>>>>
>>>>>>
>>>>>> 8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobV
>>>>>> cH
>>>>>> Ps
>>>>>> Dls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>> Archives:
>>>>>>
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabble.
>>>>>> ap
>>>>>> ar
>>>>>>
>>>>>>
>>>>>> ajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M3-
>>>>>> rw
>>>>>> SB
>>>>>>
>>>>>>
>>>>>> mptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls&s=
>>>>>> VJ
>>>>>> zL
>>>>>> TdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>
>>>>
>>>> _______________________________________________
>>>> Active4D-dev mailing list
>>>> [hidden email]
>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email]
>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
> _______________________________________________
> Active4D-dev mailing list
> [hidden email]
> http://list.aparajitaworld.com/listinfo/active4d-dev
> Archives: http://active4d-nabble.aparajitaworld.com/


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Aparajita Fishman
In reply to this post by Peter Gutbrod
Peter,

Any slowdowns yet?

All the best,

  - Aparajita

> On Apr 25, 2017, at 10:04 AM, Peter Gutbrod <[hidden email]> wrote:
>
> I had set from the beginning:
>
> NumThreads=13
> MaxThreads=97
> ThreadStack=256
> CompressResponse=1
> KeepAliveTime=5
>
> Don't think MaxThreads=97 would make a difference.
>
> Any other values to play with?
>
> Rebooted early in the morning and 12 hours later there are still no
> slowdowns, fingers crossed.
>
> Peter
>> Aparajita Fishman <mailto:[hidden email] <mailto:[hidden email]>>
>> 24. April 2017 um 17:09
>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing this to
>> 100 by default in the next release, and I am also adding code to
>> reclaim unused threads after a configurable delay.
>>
>> All the best,
>>
>> - Aparajita
>>
>>
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
> _______________________________________________
> Active4D-dev mailing list
> [hidden email] <mailto:[hidden email]>
> http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev>
> Archives: http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/>
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Aparajita Fishman
In reply to this post by Aparajita Fishman
I just did a test loading a simple page for over 12 hours with 30 simultaneous connections and the performance remained constant. So as long as you don’t reach the thread limit, I can’t see why there would be a big slowdown.

All the best,

  - Aparajita

> On Apr 25, 2017, at 10:28 AM, Aparajita Fishman <[hidden email]> wrote:
>
> If you were hitting the previous limit of 27 for MaxThreads, increasing to 97 will definitely make a difference.
>
> All the best,
>
>  - Aparajita
>
>> On Apr 25, 2017, at 10:04 AM, Peter Gutbrod <[hidden email] <mailto:[hidden email]>> wrote:
>>
>> I had set from the beginning:
>>
>> NumThreads=13
>> MaxThreads=97
>> ThreadStack=256
>> CompressResponse=1
>> KeepAliveTime=5
>>
>> Don't think MaxThreads=97 would make a difference.
>>
>> Any other values to play with?
>>
>> Rebooted early in the morning and 12 hours later there are still no
>> slowdowns, fingers crossed.
>>
>> Peter
>>> Aparajita Fishman <mailto:[hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>>
>>> 24. April 2017 um 17:09
>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing this to
>>> 100 by default in the next release, and I am also adding code to
>>> reclaim unused threads after a configurable delay.
>>>
>>> All the best,
>>>
>>> - Aparajita
>>>
>>>
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email] <mailto:[hidden email]>
>>> http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev>
>>> Archives: http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/>
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>
>> http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev> <http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev>>
>> Archives: http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/> <http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/>>
> _______________________________________________
> Active4D-dev mailing list
> [hidden email] <mailto:[hidden email]>
> http://list.aparajitaworld.com/listinfo/active4d-dev <http://list.aparajitaworld.com/listinfo/active4d-dev>
> Archives: http://active4d-nabble.aparajitaworld.com/ <http://active4d-nabble.aparajitaworld.com/>
_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Perkins, Bradley D
In reply to this post by Aparajita Fishman
The developer that did our shell upgrades responded:

This will require a code change in the shell A4D_HTTPD_RequestHandler
method, because the "A4D Execute stream request" plugin command from
Active4D v6.4r3 takes different parameters than it did in 6.1r6.


`New
$httpStatus:=A4D Execute stream request
($inSocket;$requestInfo;$headers;$responseHeaderNames;$responseHeaderValues
;$response;$requestHeaderNames;$requestHeaderValues)

`Old
$httpStatus:=A4D Execute stream request
($inSocket;$requestInfo;$responseHeaderNames;$responseHeaderValues;$respons
e)

Given that I'm not sure how we can integrate the old shell using the
latest plugins?

We've been stable under normal usage. No reported or observed slowdowns.
For some reason web client rebooted itself night before last. When it did
I didn't see anything abnormal in the logs nor did the A4D log any new
threads having been created (I added logging code to capture that).

I'll run as-is for now and continue to monitor the situation.

-- Brad

On 4/25/17, 11:29 AM, "Active4D-dev on behalf of Aparajita Fishman"
<[hidden email] on behalf of
[hidden email]> wrote:

>Yes, that’s what I propose. In the meantime I’m going to do some
>long-term stress testing here.
>
>All the best,
>
>  - Aparajita
>
>> On Apr 25, 2017, at 10:23 AM, Perkins, Bradley D <[hidden email]>
>>wrote:
>>
>> So keep latest NTK and Active4D Plugins but reinstall the the old shell
>> code? That would include using the old A4D_NTKConfig lists.
>> We can try that. It will have to wait a few weeks because of other
>>project
>> commitments.
>>
>> Thanks,
>>
>> Brad Perkins
>>
>> On 4/25/17, 9:41 AM, "Active4D-dev on behalf of Aparajita Fishman"
>> <[hidden email] on behalf of
>> [hidden email]> wrote:
>>
>>> I meant only to use the old NTK shell code. Nothing else would have to
>>> change. If your problems persist and I can’t find the solution, I’ll do
>>> it for free.
>>>
>>> All the best,
>>>
>>> - Aparajita
>>>
>>>> On Apr 25, 2017, at 7:21 AM, Perkins, Bradley D <[hidden email]>
>>>> wrote:
>>>>
>>>> Not really. I'd have to:
>>>> (a) Go back to the old server machine and 4D v14
>>>> (b) Work with institutional IT to get it back on the network
>>>>(Hostmaster
>>>> updates, Switch programming). They tightly control all public facing
>>>>IP
>>>> addresses.
>>>> (c) Identify all changed data since we went live and export/import it
>>>> back
>>>> into the v14 Database.
>>>> As noted that ran fine.
>>>>
>>>> I never considered running v14 on the Sierra machines. I was told by
>>>>4D
>>>> it
>>>> wouldn't be stable.
>>>>
>>>> If you mean staying with v15.4 and putting in the old shell code and
>>>> going
>>>> back to older NTK and Active4D versions, we could try that, but I have
>>>> other other project commitments that I must focus on for the next few
>>>> weeks and can't work on that now. We're just going to have to monitor
>>>> our
>>>> systems and live with the problems if they persist.
>>>>
>>>> One idea I did have was to compile the NTK demo database under v15.4
>>>> running same NTK and A4D versions and place that on our development
>>>> system, reasonably exercise it with AB, and see if the Active4D/NTK
>>>> server
>>>> becomes unresponsive.
>>>>
>>>> Also, I checked and we are still using the old network layer.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> -- Brad
>>>>
>>>> On 4/25/17, 7:30 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>> <[hidden email] on behalf of
>>>> [hidden email]> wrote:
>>>>
>>>>> Brad,
>>>>>
>>>>> Is it possible to roll back to the old NTK server code? I'd like to
>>>>> know
>>>>> if that solves the problem.
>>>>>
>>>>> All the best,
>>>>>
>>>>> - Aparajita
>>>>>
>>>>>> On Apr 24, 2017, at 1:53 PM, Perkins, Bradley D <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> We went from v14 to v15. We're also still running 32-bit server. I'm
>>>>>> not
>>>>>> positive but I thought the new network layer was only available in
>>>>>> 64-bit.
>>>>>> I will definitely check though. Thanks!
>>>>>>
>>>>>> Brad
>>>>>>
>>>>>> On 4/24/17, 2:31 PM, "Active4D-dev on behalf of Bellos, John"
>>>>>> <[hidden email] on behalf of
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> I forget what version of 4D you said you came from, but if it was
>>>>>>>v13
>>>>>>> or
>>>>>>> lower, try your setup with the new network layer OFF in 4D server.
>>>>>>> We've
>>>>>>> definitely seen some stability issues with the new network layer
>>>>>>>ON,
>>>>>>> using v15.4.
>>>>>>>
>>>>>>> Note that if you're using Mac 64-Bit, the new network layer is
>>>>>>>forced
>>>>>>> on
>>>>>>> and you can't turn it off. But on Windows 64 bit server, you can
>>>>>>>turn
>>>>>>> this setting off.
>>>>>>>
>>>>>>> ________________________________________
>>>>>>> From: Active4D-dev [[hidden email]] on
>>>>>>> behalf
>>>>>>> of
>>>>>>> Aparajita Fishman [[hidden email]]
>>>>>>> Sent: Monday, April 24, 2017 4:29 PM
>>>>>>> To: Active4D Developer Discussion List
>>>>>>> Subject: Re: [Active4d-dev] Active4D Slowdown then Stall (weird
>>>>>>> A4D_HTTPD_RequestHandler_* behavior)
>>>>>>>
>>>>>>> It’s entirely possible there is some flaw in the new server code.
>>>>>>>The
>>>>>>> main change I made to A4D_HTTPD_RequestHandler was the correct
>>>>>>> handling
>>>>>>> of keep-alive connections. If you find out any more details, please
>>>>>>> post
>>>>>>> them here.
>>>>>>>
>>>>>>> All the best,
>>>>>>>
>>>>>>> - Aparajita
>>>>>>>
>>>>>>>> On Apr 24, 2017, at 8:43 AM, Perkins, Bradley D
>>>>>>>><[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Peter and Aparajita,
>>>>>>>>
>>>>>>>> Thank you both for your responses. I have been testing our
>>>>>>>> development
>>>>>>>> system with a higher MaxThreads this morning. I haven't been able
>>>>>>>>to
>>>>>>>> exhaust all of the threads. I was able to crash 4D through. I'll
>>>>>>>> send
>>>>>>>> Aparajita the crash log.
>>>>>>>>
>>>>>>>> Peter, your observations are very similar to what we've seen.
>>>>>>>>Under
>>>>>>>> v14
>>>>>>>> --
>>>>>>>> and even with the broken keep alive feature -- we could run for a
>>>>>>>> long
>>>>>>>> time with no problems. Reboots were usually because I had to
>>>>>>>>reboot
>>>>>>>> the
>>>>>>>> Windows server to install MS Security Patches.
>>>>>>>>
>>>>>>>> Our production system sees a fairly light load and would normally
>>>>>>>> never
>>>>>>>> see the loads that I'm placing on it with ApacheBench unless IT is
>>>>>>>> running
>>>>>>>> a cyber scan. I'm letting it run as is with the earlier
>>>>>>>> configuration
>>>>>>>> to
>>>>>>>> see if the problem is because we've exhausted the thread pool. We
>>>>>>>> haven't
>>>>>>>> allocated an additional thread since I restarted on Saturday. Our
>>>>>>>> ThreadStack size was already set to 256 (docs show 128 as
>>>>>>>>default).
>>>>>>>> As Peter notes, It could be that the problem lies elsewhere such
>>>>>>>>as
>>>>>>>> in
>>>>>>>> in
>>>>>>>> 4D itself.
>>>>>>>>
>>>>>>>> We upgraded hardware, OS, 4D, NTK, and Active4D. That is a lot of
>>>>>>>> change.
>>>>>>>>
>>>>>>>> -- Brad
>>>>>>>>
>>>>>>>> On 4/24/17, 9:09 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>>>>> <[hidden email] on behalf of
>>>>>>>> [hidden email]> wrote:
>>>>>>>>
>>>>>>>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing
>>>>>>>>>this
>>>>>>>>> to
>>>>>>>>> 100 by default in the next release, and I am also adding code to
>>>>>>>>> reclaim
>>>>>>>>> unused threads after a configurable delay.
>>>>>>>>>
>>>>>>>>> All the best,
>>>>>>>>>
>>>>>>>>> - Aparajita
>>>>>>>>>
>>>>>>>>>> On Apr 24, 2017, at 4:51 AM, Peter Gutbrod
>>>>>>>>>><[hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Brad,
>>>>>>>>>>
>>>>>>>>>> I see something similar since I've upgraded 4D-Server to V16 and
>>>>>>>>>> NTK/A4D
>>>>>>>>>> to the newest version.
>>>>>>>>>>
>>>>>>>>>> Before the server could run für month without any issue. After
>>>>>>>>>>the
>>>>>>>>>> upgrade the server slows down within 1-2 days up the point,
>>>>>>>>>>where
>>>>>>>>>> each
>>>>>>>>>> request can take 10 seconds or even more. According to the 4D
>>>>>>>>>> admin
>>>>>>>>>> panel there are cpu or network intensive tasks running. Server
>>>>>>>>>>is
>>>>>>>>>> almost
>>>>>>>>>> idle, tons af free memory for 4D. Nothing suspicious in Windows
>>>>>>>>>> performance monitor. Rebooting 4D and the slow-down is gone -
>>>>>>>>>>for
>>>>>>>>>> the
>>>>>>>>>> next couple of hours, then it slowly reappears.
>>>>>>>>>>
>>>>>>>>>> I realized the slowdown in a JSON API. First I thought it is
>>>>>>>>>> something
>>>>>>>>>> within my A4D code, so I've calculated the time my A4D code
>>>>>>>>>>needs
>>>>>>>>>> to
>>>>>>>>>> run. But I found the slowdown happens while receiving the
>>>>>>>>>>request.
>>>>>>>>>> Once
>>>>>>>>>> A4D starts executing the code, everything runs fast, including
>>>>>>>>>> sending
>>>>>>>>>> the response.
>>>>>>>>>>
>>>>>>>>>> I've asked Rob from the pluggers, whether he had heard about
>>>>>>>>>> slowdows
>>>>>>>>>> with the new version of NTK but he wasn't aware of any problems.
>>>>>>>>>>
>>>>>>>>>> In a period, where I wasn't able to restart 4D server
>>>>>>>>>> periodically,
>>>>>>>>>> slowdown got so extreme, that 4D clients were unable to login
>>>>>>>>>>due
>>>>>>>>>> to
>>>>>>>>>> timeouts. In that state the only way to restart the 4D server
>>>>>>>>>>was
>>>>>>>>>> with a
>>>>>>>>>> force quit. That convinced me, that the culprit is within 4D V16
>>>>>>>>>> and
>>>>>>>>>> not
>>>>>>>>>> NTK or A4D and I should wait for 4D 16.1 and until then restart
>>>>>>>>>>4D
>>>>>>>>>> Server periodically. That is why I never reported the problem
>>>>>>>>>> here.
>>>>>>>>>>
>>>>>>>>>> As you have a similar problem but with 4D15.4 makes me
>>>>>>>>>>wondering,
>>>>>>>>>> whether I'm wrong and the lastest versions of NTK and A4D might
>>>>>>>>>>at
>>>>>>>>>> least
>>>>>>>>>> contribute to the problem.
>>>>>>>>>>
>>>>>>>>>> Hope you find the answer and share it with me.
>>>>>>>>>>
>>>>>>>>>> Greetings
>>>>>>>>>>
>>>>>>>>>> Peter Gutbrod
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>> [hidden email]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajit
>>>>>>>>>>aw
>>>>>>>>>> or
>>>>>>>>>> ld
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>.com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=
>>>>>>>>>>ww
>>>>>>>>>> tj
>>>>>>>>>> VR
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>q8UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4
>>>>>>>>>>G5
>>>>>>>>>> PK
>>>>>>>>>> YC
>>>>>>>>>> 6wobVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>> Archives:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dna
>>>>>>>>>>bb
>>>>>>>>>> le
>>>>>>>>>> .a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>parajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQ
>>>>>>>>>>mO
>>>>>>>>>> 8P
>>>>>>>>>> 5M
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wob
>>>>>>>>>>Vc
>>>>>>>>>> HP
>>>>>>>>>> sD
>>>>>>>>>> ls&s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Active4D-dev mailing list
>>>>>>>>> [hidden email]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajita
>>>>>>>>>wo
>>>>>>>>> rl
>>>>>>>>> d.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=ww
>>>>>>>>>tj
>>>>>>>>> VR
>>>>>>>>> q8
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5P
>>>>>>>>>KY
>>>>>>>>> C6
>>>>>>>>> wo
>>>>>>>>> bVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>> Archives:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnab
>>>>>>>>>bl
>>>>>>>>> e.
>>>>>>>>> ap
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>arajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO
>>>>>>>>>8P
>>>>>>>>> 5M
>>>>>>>>> 3-
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcH
>>>>>>>>>Ps
>>>>>>>>> Dl
>>>>>>>>> s&
>>>>>>>>> s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaw
>>>>>>>>or
>>>>>>>> ld
>>>>>>>> .c
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>om_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtj
>>>>>>>>VR
>>>>>>>> q8
>>>>>>>> UQ
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>mO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC
>>>>>>>>6w
>>>>>>>> ob
>>>>>>>> Vc
>>>>>>>> HPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>> Archives:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabb
>>>>>>>>le
>>>>>>>> .a
>>>>>>>> pa
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>rajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P
>>>>>>>>5M
>>>>>>>> 3-
>>>>>>>> rw
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>SBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsD
>>>>>>>>ls
>>>>>>>> &s
>>>>>>>> =V
>>>>>>>> JzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawo
>>>>>>>rl
>>>>>>> d.
>>>>>>> co
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>m_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVR
>>>>>>>q8
>>>>>>> UQ
>>>>>>> mO
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wo
>>>>>>>bV
>>>>>>> cH
>>>>>>> Ps
>>>>>>> Dls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>> Archives:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabbl
>>>>>>>e.
>>>>>>> ap
>>>>>>> ar
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>ajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M
>>>>>>>3-
>>>>>>> rw
>>>>>>> SB
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>mptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls&
>>>>>>>s=
>>>>>>> VJ
>>>>>>> zL
>>>>>>> TdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>
>>>> _______________________________________________
>>>> Active4D-dev mailing list
>>>> [hidden email]
>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email]
>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Aparajita Fishman
If you aren’t getting any slowdowns, then there is no need to revert to the old shell code. After doing a 12 hour stress test I’m fairly confident in the new code.

I’m in the process of improving and tuning thread management, I’ll release a new shell in the next week.

All the best,

  - Aparajita

> On Apr 27, 2017, at 1:00 PM, Perkins, Bradley D <[hidden email]> wrote:
>
> The developer that did our shell upgrades responded:
>
> This will require a code change in the shell A4D_HTTPD_RequestHandler
> method, because the "A4D Execute stream request" plugin command from
> Active4D v6.4r3 takes different parameters than it did in 6.1r6.
>
>
> `New
> $httpStatus:=A4D Execute stream request
> ($inSocket;$requestInfo;$headers;$responseHeaderNames;$responseHeaderValues
> ;$response;$requestHeaderNames;$requestHeaderValues)
>
> `Old
> $httpStatus:=A4D Execute stream request
> ($inSocket;$requestInfo;$responseHeaderNames;$responseHeaderValues;$respons
> e)
>
> Given that I'm not sure how we can integrate the old shell using the
> latest plugins?
>
> We've been stable under normal usage. No reported or observed slowdowns.
> For some reason web client rebooted itself night before last. When it did
> I didn't see anything abnormal in the logs nor did the A4D log any new
> threads having been created (I added logging code to capture that).
>
> I'll run as-is for now and continue to monitor the situation.
>
> -- Brad
>
> On 4/25/17, 11:29 AM, "Active4D-dev on behalf of Aparajita Fishman"
> <[hidden email] on behalf of
> [hidden email]> wrote:
>
>> Yes, that’s what I propose. In the meantime I’m going to do some
>> long-term stress testing here.
>>
>> All the best,
>>
>> - Aparajita
>>
>>> On Apr 25, 2017, at 10:23 AM, Perkins, Bradley D <[hidden email]>
>>> wrote:
>>>
>>> So keep latest NTK and Active4D Plugins but reinstall the the old shell
>>> code? That would include using the old A4D_NTKConfig lists.
>>> We can try that. It will have to wait a few weeks because of other
>>> project
>>> commitments.
>>>
>>> Thanks,
>>>
>>> Brad Perkins
>>>
>>> On 4/25/17, 9:41 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>> <[hidden email] on behalf of
>>> [hidden email]> wrote:
>>>
>>>> I meant only to use the old NTK shell code. Nothing else would have to
>>>> change. If your problems persist and I can’t find the solution, I’ll do
>>>> it for free.
>>>>
>>>> All the best,
>>>>
>>>> - Aparajita
>>>>
>>>>> On Apr 25, 2017, at 7:21 AM, Perkins, Bradley D <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> Not really. I'd have to:
>>>>> (a) Go back to the old server machine and 4D v14
>>>>> (b) Work with institutional IT to get it back on the network
>>>>> (Hostmaster
>>>>> updates, Switch programming). They tightly control all public facing
>>>>> IP
>>>>> addresses.
>>>>> (c) Identify all changed data since we went live and export/import it
>>>>> back
>>>>> into the v14 Database.
>>>>> As noted that ran fine.
>>>>>
>>>>> I never considered running v14 on the Sierra machines. I was told by
>>>>> 4D
>>>>> it
>>>>> wouldn't be stable.
>>>>>
>>>>> If you mean staying with v15.4 and putting in the old shell code and
>>>>> going
>>>>> back to older NTK and Active4D versions, we could try that, but I have
>>>>> other other project commitments that I must focus on for the next few
>>>>> weeks and can't work on that now. We're just going to have to monitor
>>>>> our
>>>>> systems and live with the problems if they persist.
>>>>>
>>>>> One idea I did have was to compile the NTK demo database under v15.4
>>>>> running same NTK and A4D versions and place that on our development
>>>>> system, reasonably exercise it with AB, and see if the Active4D/NTK
>>>>> server
>>>>> becomes unresponsive.
>>>>>
>>>>> Also, I checked and we are still using the old network layer.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -- Brad
>>>>>
>>>>> On 4/25/17, 7:30 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>> <[hidden email] on behalf of
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> Brad,
>>>>>>
>>>>>> Is it possible to roll back to the old NTK server code? I'd like to
>>>>>> know
>>>>>> if that solves the problem.
>>>>>>
>>>>>> All the best,
>>>>>>
>>>>>> - Aparajita
>>>>>>
>>>>>>> On Apr 24, 2017, at 1:53 PM, Perkins, Bradley D <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi John,
>>>>>>>
>>>>>>> We went from v14 to v15. We're also still running 32-bit server. I'm
>>>>>>> not
>>>>>>> positive but I thought the new network layer was only available in
>>>>>>> 64-bit.
>>>>>>> I will definitely check though. Thanks!
>>>>>>>
>>>>>>> Brad
>>>>>>>
>>>>>>> On 4/24/17, 2:31 PM, "Active4D-dev on behalf of Bellos, John"
>>>>>>> <[hidden email] on behalf of
>>>>>>> [hidden email]> wrote:
>>>>>>>
>>>>>>>> I forget what version of 4D you said you came from, but if it was
>>>>>>>> v13
>>>>>>>> or
>>>>>>>> lower, try your setup with the new network layer OFF in 4D server.
>>>>>>>> We've
>>>>>>>> definitely seen some stability issues with the new network layer
>>>>>>>> ON,
>>>>>>>> using v15.4.
>>>>>>>>
>>>>>>>> Note that if you're using Mac 64-Bit, the new network layer is
>>>>>>>> forced
>>>>>>>> on
>>>>>>>> and you can't turn it off. But on Windows 64 bit server, you can
>>>>>>>> turn
>>>>>>>> this setting off.
>>>>>>>>
>>>>>>>> ________________________________________
>>>>>>>> From: Active4D-dev [[hidden email]] on
>>>>>>>> behalf
>>>>>>>> of
>>>>>>>> Aparajita Fishman [[hidden email]]
>>>>>>>> Sent: Monday, April 24, 2017 4:29 PM
>>>>>>>> To: Active4D Developer Discussion List
>>>>>>>> Subject: Re: [Active4d-dev] Active4D Slowdown then Stall (weird
>>>>>>>> A4D_HTTPD_RequestHandler_* behavior)
>>>>>>>>
>>>>>>>> It’s entirely possible there is some flaw in the new server code.
>>>>>>>> The
>>>>>>>> main change I made to A4D_HTTPD_RequestHandler was the correct
>>>>>>>> handling
>>>>>>>> of keep-alive connections. If you find out any more details, please
>>>>>>>> post
>>>>>>>> them here.
>>>>>>>>
>>>>>>>> All the best,
>>>>>>>>
>>>>>>>> - Aparajita
>>>>>>>>
>>>>>>>>> On Apr 24, 2017, at 8:43 AM, Perkins, Bradley D
>>>>>>>>> <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Peter and Aparajita,
>>>>>>>>>
>>>>>>>>> Thank you both for your responses. I have been testing our
>>>>>>>>> development
>>>>>>>>> system with a higher MaxThreads this morning. I haven't been able
>>>>>>>>> to
>>>>>>>>> exhaust all of the threads. I was able to crash 4D through. I'll
>>>>>>>>> send
>>>>>>>>> Aparajita the crash log.
>>>>>>>>>
>>>>>>>>> Peter, your observations are very similar to what we've seen.
>>>>>>>>> Under
>>>>>>>>> v14
>>>>>>>>> --
>>>>>>>>> and even with the broken keep alive feature -- we could run for a
>>>>>>>>> long
>>>>>>>>> time with no problems. Reboots were usually because I had to
>>>>>>>>> reboot
>>>>>>>>> the
>>>>>>>>> Windows server to install MS Security Patches.
>>>>>>>>>
>>>>>>>>> Our production system sees a fairly light load and would normally
>>>>>>>>> never
>>>>>>>>> see the loads that I'm placing on it with ApacheBench unless IT is
>>>>>>>>> running
>>>>>>>>> a cyber scan. I'm letting it run as is with the earlier
>>>>>>>>> configuration
>>>>>>>>> to
>>>>>>>>> see if the problem is because we've exhausted the thread pool. We
>>>>>>>>> haven't
>>>>>>>>> allocated an additional thread since I restarted on Saturday. Our
>>>>>>>>> ThreadStack size was already set to 256 (docs show 128 as
>>>>>>>>> default).
>>>>>>>>> As Peter notes, It could be that the problem lies elsewhere such
>>>>>>>>> as
>>>>>>>>> in
>>>>>>>>> in
>>>>>>>>> 4D itself.
>>>>>>>>>
>>>>>>>>> We upgraded hardware, OS, 4D, NTK, and Active4D. That is a lot of
>>>>>>>>> change.
>>>>>>>>>
>>>>>>>>> -- Brad
>>>>>>>>>
>>>>>>>>> On 4/24/17, 9:09 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>>>>>> <[hidden email] on behalf of
>>>>>>>>> [hidden email]> wrote:
>>>>>>>>>
>>>>>>>>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing
>>>>>>>>>> this
>>>>>>>>>> to
>>>>>>>>>> 100 by default in the next release, and I am also adding code to
>>>>>>>>>> reclaim
>>>>>>>>>> unused threads after a configurable delay.
>>>>>>>>>>
>>>>>>>>>> All the best,
>>>>>>>>>>
>>>>>>>>>> - Aparajita
>>>>>>>>>>
>>>>>>>>>>> On Apr 24, 2017, at 4:51 AM, Peter Gutbrod
>>>>>>>>>>> <[hidden email]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Brad,
>>>>>>>>>>>
>>>>>>>>>>> I see something similar since I've upgraded 4D-Server to V16 and
>>>>>>>>>>> NTK/A4D
>>>>>>>>>>> to the newest version.
>>>>>>>>>>>
>>>>>>>>>>> Before the server could run für month without any issue. After
>>>>>>>>>>> the
>>>>>>>>>>> upgrade the server slows down within 1-2 days up the point,
>>>>>>>>>>> where
>>>>>>>>>>> each
>>>>>>>>>>> request can take 10 seconds or even more. According to the 4D
>>>>>>>>>>> admin
>>>>>>>>>>> panel there are cpu or network intensive tasks running. Server
>>>>>>>>>>> is
>>>>>>>>>>> almost
>>>>>>>>>>> idle, tons af free memory for 4D. Nothing suspicious in Windows
>>>>>>>>>>> performance monitor. Rebooting 4D and the slow-down is gone -
>>>>>>>>>>> for
>>>>>>>>>>> the
>>>>>>>>>>> next couple of hours, then it slowly reappears.
>>>>>>>>>>>
>>>>>>>>>>> I realized the slowdown in a JSON API. First I thought it is
>>>>>>>>>>> something
>>>>>>>>>>> within my A4D code, so I've calculated the time my A4D code
>>>>>>>>>>> needs
>>>>>>>>>>> to
>>>>>>>>>>> run. But I found the slowdown happens while receiving the
>>>>>>>>>>> request.
>>>>>>>>>>> Once
>>>>>>>>>>> A4D starts executing the code, everything runs fast, including
>>>>>>>>>>> sending
>>>>>>>>>>> the response.
>>>>>>>>>>>
>>>>>>>>>>> I've asked Rob from the pluggers, whether he had heard about
>>>>>>>>>>> slowdows
>>>>>>>>>>> with the new version of NTK but he wasn't aware of any problems.
>>>>>>>>>>>
>>>>>>>>>>> In a period, where I wasn't able to restart 4D server
>>>>>>>>>>> periodically,
>>>>>>>>>>> slowdown got so extreme, that 4D clients were unable to login
>>>>>>>>>>> due
>>>>>>>>>>> to
>>>>>>>>>>> timeouts. In that state the only way to restart the 4D server
>>>>>>>>>>> was
>>>>>>>>>>> with a
>>>>>>>>>>> force quit. That convinced me, that the culprit is within 4D V16
>>>>>>>>>>> and
>>>>>>>>>>> not
>>>>>>>>>>> NTK or A4D and I should wait for 4D 16.1 and until then restart
>>>>>>>>>>> 4D
>>>>>>>>>>> Server periodically. That is why I never reported the problem
>>>>>>>>>>> here.
>>>>>>>>>>>
>>>>>>>>>>> As you have a similar problem but with 4D15.4 makes me
>>>>>>>>>>> wondering,
>>>>>>>>>>> whether I'm wrong and the lastest versions of NTK and A4D might
>>>>>>>>>>> at
>>>>>>>>>>> least
>>>>>>>>>>> contribute to the problem.
>>>>>>>>>>>
>>>>>>>>>>> Hope you find the answer and share it with me.
>>>>>>>>>>>
>>>>>>>>>>> Greetings
>>>>>>>>>>>
>>>>>>>>>>> Peter Gutbrod
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>>> [hidden email]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajit
>>>>>>>>>>> aw
>>>>>>>>>>> or
>>>>>>>>>>> ld
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> .com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=
>>>>>>>>>>> ww
>>>>>>>>>>> tj
>>>>>>>>>>> VR
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> q8UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4
>>>>>>>>>>> G5
>>>>>>>>>>> PK
>>>>>>>>>>> YC
>>>>>>>>>>> 6wobVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>>> Archives:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dna
>>>>>>>>>>> bb
>>>>>>>>>>> le
>>>>>>>>>>> .a
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> parajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQ
>>>>>>>>>>> mO
>>>>>>>>>>> 8P
>>>>>>>>>>> 5M
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wob
>>>>>>>>>>> Vc
>>>>>>>>>>> HP
>>>>>>>>>>> sD
>>>>>>>>>>> ls&s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>> [hidden email]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajita
>>>>>>>>>> wo
>>>>>>>>>> rl
>>>>>>>>>> d.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=ww
>>>>>>>>>> tj
>>>>>>>>>> VR
>>>>>>>>>> q8
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5P
>>>>>>>>>> KY
>>>>>>>>>> C6
>>>>>>>>>> wo
>>>>>>>>>> bVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>> Archives:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnab
>>>>>>>>>> bl
>>>>>>>>>> e.
>>>>>>>>>> ap
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> arajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO
>>>>>>>>>> 8P
>>>>>>>>>> 5M
>>>>>>>>>> 3-
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcH
>>>>>>>>>> Ps
>>>>>>>>>> Dl
>>>>>>>>>> s&
>>>>>>>>>> s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Active4D-dev mailing list
>>>>>>>>> [hidden email]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitaw
>>>>>>>>> or
>>>>>>>>> ld
>>>>>>>>> .c
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> om_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtj
>>>>>>>>> VR
>>>>>>>>> q8
>>>>>>>>> UQ
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> mO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC
>>>>>>>>> 6w
>>>>>>>>> ob
>>>>>>>>> Vc
>>>>>>>>> HPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>> Archives:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabb
>>>>>>>>> le
>>>>>>>>> .a
>>>>>>>>> pa
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> rajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P
>>>>>>>>> 5M
>>>>>>>>> 3-
>>>>>>>>> rw
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> SBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsD
>>>>>>>>> ls
>>>>>>>>> &s
>>>>>>>>> =V
>>>>>>>>> JzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajitawo
>>>>>>>> rl
>>>>>>>> d.
>>>>>>>> co
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> m_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVR
>>>>>>>> q8
>>>>>>>> UQ
>>>>>>>> mO
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wo
>>>>>>>> bV
>>>>>>>> cH
>>>>>>>> Ps
>>>>>>>> Dls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>> Archives:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnabbl
>>>>>>>> e.
>>>>>>>> ap
>>>>>>>> ar
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P5M
>>>>>>>> 3-
>>>>>>>> rw
>>>>>>>> SB
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> mptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDls&
>>>>>>>> s=
>>>>>>>> VJ
>>>>>>>> zL
>>>>>>>> TdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>
>>>>
>>>> _______________________________________________
>>>> Active4D-dev mailing list
>>>> [hidden email]
>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email]
>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
> _______________________________________________
> Active4D-dev mailing list
> [hidden email]
> http://list.aparajitaworld.com/listinfo/active4d-dev
> Archives: http://active4d-nabble.aparajitaworld.com/


_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
Reply | Threaded
Open this post in threaded view
|

Re: Active4D Slowdown then Stall (weird A4D_HTTPD_RequestHandler_* behavior)

Perkins, Bradley D
Thanks. I look forward to seeing the new shell.

Brad

On 4/27/17, 2:03 PM, "Active4D-dev on behalf of Aparajita Fishman"
<[hidden email] on behalf of
[hidden email]> wrote:

>If you aren’t getting any slowdowns, then there is no need to revert to
>the old shell code. After doing a 12 hour stress test I’m fairly
>confident in the new code.
>
>I’m in the process of improving and tuning thread management, I’ll
>release a new shell in the next week.
>
>All the best,
>
>  - Aparajita
>
>> On Apr 27, 2017, at 1:00 PM, Perkins, Bradley D <[hidden email]>
>>wrote:
>>
>> The developer that did our shell upgrades responded:
>>
>> This will require a code change in the shell A4D_HTTPD_RequestHandler
>> method, because the "A4D Execute stream request" plugin command from
>> Active4D v6.4r3 takes different parameters than it did in 6.1r6.
>>
>>
>> `New
>> $httpStatus:=A4D Execute stream request
>>
>>($inSocket;$requestInfo;$headers;$responseHeaderNames;$responseHeaderValu
>>es
>> ;$response;$requestHeaderNames;$requestHeaderValues)
>>
>> `Old
>> $httpStatus:=A4D Execute stream request
>>
>>($inSocket;$requestInfo;$responseHeaderNames;$responseHeaderValues;$respo
>>ns
>> e)
>>
>> Given that I'm not sure how we can integrate the old shell using the
>> latest plugins?
>>
>> We've been stable under normal usage. No reported or observed slowdowns.
>> For some reason web client rebooted itself night before last. When it
>>did
>> I didn't see anything abnormal in the logs nor did the A4D log any new
>> threads having been created (I added logging code to capture that).
>>
>> I'll run as-is for now and continue to monitor the situation.
>>
>> -- Brad
>>
>> On 4/25/17, 11:29 AM, "Active4D-dev on behalf of Aparajita Fishman"
>> <[hidden email] on behalf of
>> [hidden email]> wrote:
>>
>>> Yes, that’s what I propose. In the meantime I’m going to do some
>>> long-term stress testing here.
>>>
>>> All the best,
>>>
>>> - Aparajita
>>>
>>>> On Apr 25, 2017, at 10:23 AM, Perkins, Bradley D <[hidden email]>
>>>> wrote:
>>>>
>>>> So keep latest NTK and Active4D Plugins but reinstall the the old
>>>>shell
>>>> code? That would include using the old A4D_NTKConfig lists.
>>>> We can try that. It will have to wait a few weeks because of other
>>>> project
>>>> commitments.
>>>>
>>>> Thanks,
>>>>
>>>> Brad Perkins
>>>>
>>>> On 4/25/17, 9:41 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>> <[hidden email] on behalf of
>>>> [hidden email]> wrote:
>>>>
>>>>> I meant only to use the old NTK shell code. Nothing else would have
>>>>>to
>>>>> change. If your problems persist and I can’t find the solution, I’ll
>>>>>do
>>>>> it for free.
>>>>>
>>>>> All the best,
>>>>>
>>>>> - Aparajita
>>>>>
>>>>>> On Apr 25, 2017, at 7:21 AM, Perkins, Bradley D <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>> Not really. I'd have to:
>>>>>> (a) Go back to the old server machine and 4D v14
>>>>>> (b) Work with institutional IT to get it back on the network
>>>>>> (Hostmaster
>>>>>> updates, Switch programming). They tightly control all public facing
>>>>>> IP
>>>>>> addresses.
>>>>>> (c) Identify all changed data since we went live and export/import
>>>>>>it
>>>>>> back
>>>>>> into the v14 Database.
>>>>>> As noted that ran fine.
>>>>>>
>>>>>> I never considered running v14 on the Sierra machines. I was told by
>>>>>> 4D
>>>>>> it
>>>>>> wouldn't be stable.
>>>>>>
>>>>>> If you mean staying with v15.4 and putting in the old shell code and
>>>>>> going
>>>>>> back to older NTK and Active4D versions, we could try that, but I
>>>>>>have
>>>>>> other other project commitments that I must focus on for the next
>>>>>>few
>>>>>> weeks and can't work on that now. We're just going to have to
>>>>>>monitor
>>>>>> our
>>>>>> systems and live with the problems if they persist.
>>>>>>
>>>>>> One idea I did have was to compile the NTK demo database under v15.4
>>>>>> running same NTK and A4D versions and place that on our development
>>>>>> system, reasonably exercise it with AB, and see if the Active4D/NTK
>>>>>> server
>>>>>> becomes unresponsive.
>>>>>>
>>>>>> Also, I checked and we are still using the old network layer.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -- Brad
>>>>>>
>>>>>> On 4/25/17, 7:30 AM, "Active4D-dev on behalf of Aparajita Fishman"
>>>>>> <[hidden email] on behalf of
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> Brad,
>>>>>>>
>>>>>>> Is it possible to roll back to the old NTK server code? I'd like to
>>>>>>> know
>>>>>>> if that solves the problem.
>>>>>>>
>>>>>>> All the best,
>>>>>>>
>>>>>>> - Aparajita
>>>>>>>
>>>>>>>> On Apr 24, 2017, at 1:53 PM, Perkins, Bradley D
>>>>>>>><[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi John,
>>>>>>>>
>>>>>>>> We went from v14 to v15. We're also still running 32-bit server.
>>>>>>>>I'm
>>>>>>>> not
>>>>>>>> positive but I thought the new network layer was only available in
>>>>>>>> 64-bit.
>>>>>>>> I will definitely check though. Thanks!
>>>>>>>>
>>>>>>>> Brad
>>>>>>>>
>>>>>>>> On 4/24/17, 2:31 PM, "Active4D-dev on behalf of Bellos, John"
>>>>>>>> <[hidden email] on behalf of
>>>>>>>> [hidden email]> wrote:
>>>>>>>>
>>>>>>>>> I forget what version of 4D you said you came from, but if it was
>>>>>>>>> v13
>>>>>>>>> or
>>>>>>>>> lower, try your setup with the new network layer OFF in 4D
>>>>>>>>>server.
>>>>>>>>> We've
>>>>>>>>> definitely seen some stability issues with the new network layer
>>>>>>>>> ON,
>>>>>>>>> using v15.4.
>>>>>>>>>
>>>>>>>>> Note that if you're using Mac 64-Bit, the new network layer is
>>>>>>>>> forced
>>>>>>>>> on
>>>>>>>>> and you can't turn it off. But on Windows 64 bit server, you can
>>>>>>>>> turn
>>>>>>>>> this setting off.
>>>>>>>>>
>>>>>>>>> ________________________________________
>>>>>>>>> From: Active4D-dev [[hidden email]] on
>>>>>>>>> behalf
>>>>>>>>> of
>>>>>>>>> Aparajita Fishman [[hidden email]]
>>>>>>>>> Sent: Monday, April 24, 2017 4:29 PM
>>>>>>>>> To: Active4D Developer Discussion List
>>>>>>>>> Subject: Re: [Active4d-dev] Active4D Slowdown then Stall (weird
>>>>>>>>> A4D_HTTPD_RequestHandler_* behavior)
>>>>>>>>>
>>>>>>>>> It’s entirely possible there is some flaw in the new server code.
>>>>>>>>> The
>>>>>>>>> main change I made to A4D_HTTPD_RequestHandler was the correct
>>>>>>>>> handling
>>>>>>>>> of keep-alive connections. If you find out any more details,
>>>>>>>>>please
>>>>>>>>> post
>>>>>>>>> them here.
>>>>>>>>>
>>>>>>>>> All the best,
>>>>>>>>>
>>>>>>>>> - Aparajita
>>>>>>>>>
>>>>>>>>>> On Apr 24, 2017, at 8:43 AM, Perkins, Bradley D
>>>>>>>>>> <[hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Peter and Aparajita,
>>>>>>>>>>
>>>>>>>>>> Thank you both for your responses. I have been testing our
>>>>>>>>>> development
>>>>>>>>>> system with a higher MaxThreads this morning. I haven't been
>>>>>>>>>>able
>>>>>>>>>> to
>>>>>>>>>> exhaust all of the threads. I was able to crash 4D through. I'll
>>>>>>>>>> send
>>>>>>>>>> Aparajita the crash log.
>>>>>>>>>>
>>>>>>>>>> Peter, your observations are very similar to what we've seen.
>>>>>>>>>> Under
>>>>>>>>>> v14
>>>>>>>>>> --
>>>>>>>>>> and even with the broken keep alive feature -- we could run for
>>>>>>>>>>a
>>>>>>>>>> long
>>>>>>>>>> time with no problems. Reboots were usually because I had to
>>>>>>>>>> reboot
>>>>>>>>>> the
>>>>>>>>>> Windows server to install MS Security Patches.
>>>>>>>>>>
>>>>>>>>>> Our production system sees a fairly light load and would
>>>>>>>>>>normally
>>>>>>>>>> never
>>>>>>>>>> see the loads that I'm placing on it with ApacheBench unless IT
>>>>>>>>>>is
>>>>>>>>>> running
>>>>>>>>>> a cyber scan. I'm letting it run as is with the earlier
>>>>>>>>>> configuration
>>>>>>>>>> to
>>>>>>>>>> see if the problem is because we've exhausted the thread pool.
>>>>>>>>>>We
>>>>>>>>>> haven't
>>>>>>>>>> allocated an additional thread since I restarted on Saturday.
>>>>>>>>>>Our
>>>>>>>>>> ThreadStack size was already set to 256 (docs show 128 as
>>>>>>>>>> default).
>>>>>>>>>> As Peter notes, It could be that the problem lies elsewhere such
>>>>>>>>>> as
>>>>>>>>>> in
>>>>>>>>>> in
>>>>>>>>>> 4D itself.
>>>>>>>>>>
>>>>>>>>>> We upgraded hardware, OS, 4D, NTK, and Active4D. That is a lot
>>>>>>>>>>of
>>>>>>>>>> change.
>>>>>>>>>>
>>>>>>>>>> -- Brad
>>>>>>>>>>
>>>>>>>>>> On 4/24/17, 9:09 AM, "Active4D-dev on behalf of Aparajita
>>>>>>>>>>Fishman"
>>>>>>>>>> <[hidden email] on behalf of
>>>>>>>>>> [hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Increase MaxThreads in the A4D_NTKConfig list. I am increasing
>>>>>>>>>>> this
>>>>>>>>>>> to
>>>>>>>>>>> 100 by default in the next release, and I am also adding code
>>>>>>>>>>>to
>>>>>>>>>>> reclaim
>>>>>>>>>>> unused threads after a configurable delay.
>>>>>>>>>>>
>>>>>>>>>>> All the best,
>>>>>>>>>>>
>>>>>>>>>>> - Aparajita
>>>>>>>>>>>
>>>>>>>>>>>> On Apr 24, 2017, at 4:51 AM, Peter Gutbrod
>>>>>>>>>>>> <[hidden email]>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Brad,
>>>>>>>>>>>>
>>>>>>>>>>>> I see something similar since I've upgraded 4D-Server to V16
>>>>>>>>>>>>and
>>>>>>>>>>>> NTK/A4D
>>>>>>>>>>>> to the newest version.
>>>>>>>>>>>>
>>>>>>>>>>>> Before the server could run für month without any issue. After
>>>>>>>>>>>> the
>>>>>>>>>>>> upgrade the server slows down within 1-2 days up the point,
>>>>>>>>>>>> where
>>>>>>>>>>>> each
>>>>>>>>>>>> request can take 10 seconds or even more. According to the 4D
>>>>>>>>>>>> admin
>>>>>>>>>>>> panel there are cpu or network intensive tasks running. Server
>>>>>>>>>>>> is
>>>>>>>>>>>> almost
>>>>>>>>>>>> idle, tons af free memory for 4D. Nothing suspicious in
>>>>>>>>>>>>Windows
>>>>>>>>>>>> performance monitor. Rebooting 4D and the slow-down is gone -
>>>>>>>>>>>> for
>>>>>>>>>>>> the
>>>>>>>>>>>> next couple of hours, then it slowly reappears.
>>>>>>>>>>>>
>>>>>>>>>>>> I realized the slowdown in a JSON API. First I thought it is
>>>>>>>>>>>> something
>>>>>>>>>>>> within my A4D code, so I've calculated the time my A4D code
>>>>>>>>>>>> needs
>>>>>>>>>>>> to
>>>>>>>>>>>> run. But I found the slowdown happens while receiving the
>>>>>>>>>>>> request.
>>>>>>>>>>>> Once
>>>>>>>>>>>> A4D starts executing the code, everything runs fast, including
>>>>>>>>>>>> sending
>>>>>>>>>>>> the response.
>>>>>>>>>>>>
>>>>>>>>>>>> I've asked Rob from the pluggers, whether he had heard about
>>>>>>>>>>>> slowdows
>>>>>>>>>>>> with the new version of NTK but he wasn't aware of any
>>>>>>>>>>>>problems.
>>>>>>>>>>>>
>>>>>>>>>>>> In a period, where I wasn't able to restart 4D server
>>>>>>>>>>>> periodically,
>>>>>>>>>>>> slowdown got so extreme, that 4D clients were unable to login
>>>>>>>>>>>> due
>>>>>>>>>>>> to
>>>>>>>>>>>> timeouts. In that state the only way to restart the 4D server
>>>>>>>>>>>> was
>>>>>>>>>>>> with a
>>>>>>>>>>>> force quit. That convinced me, that the culprit is within 4D
>>>>>>>>>>>>V16
>>>>>>>>>>>> and
>>>>>>>>>>>> not
>>>>>>>>>>>> NTK or A4D and I should wait for 4D 16.1 and until then
>>>>>>>>>>>>restart
>>>>>>>>>>>> 4D
>>>>>>>>>>>> Server periodically. That is why I never reported the problem
>>>>>>>>>>>> here.
>>>>>>>>>>>>
>>>>>>>>>>>> As you have a similar problem but with 4D15.4 makes me
>>>>>>>>>>>> wondering,
>>>>>>>>>>>> whether I'm wrong and the lastest versions of NTK and A4D
>>>>>>>>>>>>might
>>>>>>>>>>>> at
>>>>>>>>>>>> least
>>>>>>>>>>>> contribute to the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope you find the answer and share it with me.
>>>>>>>>>>>>
>>>>>>>>>>>> Greetings
>>>>>>>>>>>>
>>>>>>>>>>>> Peter Gutbrod
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>>>> [hidden email]
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparaj
>>>>>>>>>>>>it
>>>>>>>>>>>> aw
>>>>>>>>>>>> or
>>>>>>>>>>>> ld
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>.com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&
>>>>>>>>>>>>r=
>>>>>>>>>>>> ww
>>>>>>>>>>>> tj
>>>>>>>>>>>> VR
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>q8UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69
>>>>>>>>>>>>p4
>>>>>>>>>>>> G5
>>>>>>>>>>>> PK
>>>>>>>>>>>> YC
>>>>>>>>>>>> 6wobVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>>>> Archives:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2D
>>>>>>>>>>>>na
>>>>>>>>>>>> bb
>>>>>>>>>>>> le
>>>>>>>>>>>> .a
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>parajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8
>>>>>>>>>>>>UQ
>>>>>>>>>>>> mO
>>>>>>>>>>>> 8P
>>>>>>>>>>>> 5M
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6w
>>>>>>>>>>>>ob
>>>>>>>>>>>> Vc
>>>>>>>>>>>> HP
>>>>>>>>>>>> sD
>>>>>>>>>>>> ls&s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>>> [hidden email]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparaji
>>>>>>>>>>>ta
>>>>>>>>>>> wo
>>>>>>>>>>> rl
>>>>>>>>>>> d.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>com_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=
>>>>>>>>>>>ww
>>>>>>>>>>> tj
>>>>>>>>>>> VR
>>>>>>>>>>> q8
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>UQmO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G
>>>>>>>>>>>5P
>>>>>>>>>>> KY
>>>>>>>>>>> C6
>>>>>>>>>>> wo
>>>>>>>>>>> bVcHPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>>> Archives:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dn
>>>>>>>>>>>ab
>>>>>>>>>>> bl
>>>>>>>>>>> e.
>>>>>>>>>>> ap
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>arajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQ
>>>>>>>>>>>mO
>>>>>>>>>>> 8P
>>>>>>>>>>> 5M
>>>>>>>>>>> 3-
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobV
>>>>>>>>>>>cH
>>>>>>>>>>> Ps
>>>>>>>>>>> Dl
>>>>>>>>>>> s&
>>>>>>>>>>> s=VJzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Active4D-dev mailing list
>>>>>>>>>> [hidden email]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajit
>>>>>>>>>>aw
>>>>>>>>>> or
>>>>>>>>>> ld
>>>>>>>>>> .c
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>om_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=ww
>>>>>>>>>>tj
>>>>>>>>>> VR
>>>>>>>>>> q8
>>>>>>>>>> UQ
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>mO8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PK
>>>>>>>>>>YC
>>>>>>>>>> 6w
>>>>>>>>>> ob
>>>>>>>>>> Vc
>>>>>>>>>> HPsDls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>>> Archives:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dna
>>>>>>>>>>bb
>>>>>>>>>> le
>>>>>>>>>> .a
>>>>>>>>>> pa
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>rajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO
>>>>>>>>>>8P
>>>>>>>>>> 5M
>>>>>>>>>> 3-
>>>>>>>>>> rw
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>SBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHP
>>>>>>>>>>sD
>>>>>>>>>> ls
>>>>>>>>>> &s
>>>>>>>>>> =V
>>>>>>>>>> JzLTdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Active4D-dev mailing list
>>>>>>>>> [hidden email]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__list.aparajita
>>>>>>>>>wo
>>>>>>>>> rl
>>>>>>>>> d.
>>>>>>>>> co
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>m_listinfo_active4d-2Ddev&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtj
>>>>>>>>>VR
>>>>>>>>> q8
>>>>>>>>> UQ
>>>>>>>>> mO
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>8P5M3-rwSBmptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6
>>>>>>>>>wo
>>>>>>>>> bV
>>>>>>>>> cH
>>>>>>>>> Ps
>>>>>>>>> Dls&s=TJmubZ_4H7iiYRramtlMy84fx1lyLkuOJfA391x78cs&e=
>>>>>>>>> Archives:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>https://urldefense.proofpoint.com/v2/url?u=http-3A__active4d-2Dnab
>>>>>>>>>bl
>>>>>>>>> e.
>>>>>>>>> ap
>>>>>>>>> ar
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>ajitaworld.com_&d=DwIGaQ&c=2do6VJGs3LvEOe4OFFM1bA&r=wwtjVRq8UQmO8P
>>>>>>>>>5M
>>>>>>>>> 3-
>>>>>>>>> rw
>>>>>>>>> SB
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>mptOUUgRJNGBQlLDclI30&m=WKACtLPtkhCbwM0QslQxK69p4G5PKYC6wobVcHPsDl
>>>>>>>>>s&
>>>>>>>>> s=
>>>>>>>>> VJ
>>>>>>>>> zL
>>>>>>>>> TdVZBbjHRZH1usEIxPXoRTc4FrJF5QANN_xTdRk&e=
>>>>>>>>> _______________________________________________
>>>>>>>>> Active4D-dev mailing list
>>>>>>>>> [hidden email]
>>>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Active4D-dev mailing list
>>>>>>>> [hidden email]
>>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Active4D-dev mailing list
>>>>>>> [hidden email]
>>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>>
>>>>>> _______________________________________________
>>>>>> Active4D-dev mailing list
>>>>>> [hidden email]
>>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Active4D-dev mailing list
>>>>> [hidden email]
>>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>>
>>>> _______________________________________________
>>>> Active4D-dev mailing list
>>>> [hidden email]
>>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>>
>>>
>>> _______________________________________________
>>> Active4D-dev mailing list
>>> [hidden email]
>>> http://list.aparajitaworld.com/listinfo/active4d-dev
>>> Archives: http://active4d-nabble.aparajitaworld.com/
>>
>> _______________________________________________
>> Active4D-dev mailing list
>> [hidden email]
>> http://list.aparajitaworld.com/listinfo/active4d-dev
>> Archives: http://active4d-nabble.aparajitaworld.com/
>
>
>_______________________________________________
>Active4D-dev mailing list
>[hidden email]
>http://list.aparajitaworld.com/listinfo/active4d-dev
>Archives: http://active4d-nabble.aparajitaworld.com/

_______________________________________________
Active4D-dev mailing list
[hidden email]
http://list.aparajitaworld.com/listinfo/active4d-dev
Archives: http://active4d-nabble.aparajitaworld.com/
12