Quantcast

Re: FW: LucidDB questions

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FW: LucidDB questions

Hunter Payne
Julian,
   I believe this is an issue with any non-persistent connection
transport protocol which I believe the HTTP transport is.  In the
protocol we use, one or more persistent connections keep us informed
when a connection is broken and the relevant peers are cleaned up on
network failure.  We even have coverage for this in our unit test suite.

   However, for a non-persistent network transport mechanism there is no
notification except for a failed client driver operation (which should
throw some kind of SQLException) which should signal the calling code to
clean up its resources (by calling close()).  In the case of the
PingCommand, its created by a thread inside of the VJDBC package that
evidently has a problem cleaning itself up.  In other words, the
internal thread inside VJDBC doesn't call VJDBC correctly w.r.t.
handling exceptions.  This is probably OK as VJDBC is expecting you to
discover this yourself through other calls to the connection or its
statements.  If this doesn't happen, the internal thread happily
continues to send these types of messages to the log until you clean up
the connection yourself.

   I can make a fix to VJDBC for this if you like.  We can do this by
not using the DecoratedCommandSink as a wrapper around the client's HTTP
command sink object.  Or simply make a property to determine if to start
the KeepAliveTimerTask and turn it off in the HTTP client driver.  Or
more dangerously make the KeepAliveTimerTask close the connection on an
exception but this is tricky as it could fool calling code because it
introduces the possibility of a connection closing right out from under
a working statement.  The cleanup behavior probably would depend upon
the specific type of transport which introduces complications.

Questions?  Comments?

Hunter

On Wed, 2010-03-31 at 12:45 -0700, Julian Hyde wrote:

> Is this an issue for us?
>
> Can you assist the LucidDB list.
>
> -----Original Message-----
> From: John V. Sichi [mailto:[hidden email]]
> Sent: Wednesday, March 31, 2010 12:50 AM
> To: Mailing list for users of LucidDB
> Subject: Re: [luciddb-users] LucidDB questions
>
> I left a server running with a connection open, and I was able to
> reproduce your problem.  In fact, the connection seems to have been
> cleaned up after a timeout on the server side, so that further commands
> from it fail with a similar exception.  The ping must be from the client
> side.  We had a similar problem in the past with RMI, which is when we
> added the connectionTimeoutMillis parameter (default 1 day), but this
> must be something different for HTTP.
>
> Thanks for the report; I'll investigate the relevant VJDBC settings.
>
> JVS
>
> haawker wrote:
> > Periodically I see this SEVERE error in the Trace.log. It appears to
> happen
> > at a fixed 20 seconds interval.  
> >
> > Mar 29, 2010 1:35:06 PM de.simplicit.vjdbc.server.command.CommandProcessor
> > process
> > SEVERE: Unknown connection entry 23 for command PingCommand
> >
> > I have closed all client connections, checked for zombies via ps, and even
> > restarted the server to try and clear it out but nothing seems to work.
> > According to the dba_sql_statements table there are no rogue statement
> being
> > processed either.
> >
> > What does this mean and how do stop it from filling up my logs?
>
>
> ----------------------------------------------------------------------------
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>



------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FW: LucidDB questions

John Sichi
Administrator
For the specific issue of the existing connection timeout parameter not
being honored as it was for RMI, I have a fix which I will check in.

http://issues.eigenbase.org/browse/FRG-395

For the general issue of how the timeout is implemented, I agree with
Hunter's assessment.  For the solution, I'd say that when the server
times out a connection, the next ping from that client should return a
failure to the client, and the client then sets the closed state
(causing further calls to fail immediately) and stops pinging.  That's
the ideal, but as Hunter says, we have to be careful about how we handle
calls already in progress (i.e. be careful about concurrency, and make
the diagnostics as clear as possible so that connection timeouts don't
get misinterpreted as some other kind of failure).

JVS

Hunter Payne wrote:

> Julian,
>    I believe this is an issue with any non-persistent connection
> transport protocol which I believe the HTTP transport is.  In the
> protocol we use, one or more persistent connections keep us informed
> when a connection is broken and the relevant peers are cleaned up on
> network failure.  We even have coverage for this in our unit test suite.
>
>    However, for a non-persistent network transport mechanism there is no
> notification except for a failed client driver operation (which should
> throw some kind of SQLException) which should signal the calling code to
> clean up its resources (by calling close()).  In the case of the
> PingCommand, its created by a thread inside of the VJDBC package that
> evidently has a problem cleaning itself up.  In other words, the
> internal thread inside VJDBC doesn't call VJDBC correctly w.r.t.
> handling exceptions.  This is probably OK as VJDBC is expecting you to
> discover this yourself through other calls to the connection or its
> statements.  If this doesn't happen, the internal thread happily
> continues to send these types of messages to the log until you clean up
> the connection yourself.
>
>    I can make a fix to VJDBC for this if you like.  We can do this by
> not using the DecoratedCommandSink as a wrapper around the client's HTTP
> command sink object.  Or simply make a property to determine if to start
> the KeepAliveTimerTask and turn it off in the HTTP client driver.  Or
> more dangerously make the KeepAliveTimerTask close the connection on an
> exception but this is tricky as it could fool calling code because it
> introduces the possibility of a connection closing right out from under
> a working statement.  The cleanup behavior probably would depend upon
> the specific type of transport which introduces complications.
>
> Questions?  Comments?
>
> Hunter
>
> On Wed, 2010-03-31 at 12:45 -0700, Julian Hyde wrote:
>> Is this an issue for us?
>>
>> Can you assist the LucidDB list.
>>
>> -----Original Message-----
>> From: John V. Sichi [mailto:[hidden email]]
>> Sent: Wednesday, March 31, 2010 12:50 AM
>> To: Mailing list for users of LucidDB
>> Subject: Re: [luciddb-users] LucidDB questions
>>
>> I left a server running with a connection open, and I was able to
>> reproduce your problem.  In fact, the connection seems to have been
>> cleaned up after a timeout on the server side, so that further commands
>> from it fail with a similar exception.  The ping must be from the client
>> side.  We had a similar problem in the past with RMI, which is when we
>> added the connectionTimeoutMillis parameter (default 1 day), but this
>> must be something different for HTTP.
>>
>> Thanks for the report; I'll investigate the relevant VJDBC settings.
>>
>> JVS
>>
>> haawker wrote:
>>> Periodically I see this SEVERE error in the Trace.log. It appears to
>> happen
>>> at a fixed 20 seconds interval.  
>>>
>>> Mar 29, 2010 1:35:06 PM de.simplicit.vjdbc.server.command.CommandProcessor
>>> process
>>> SEVERE: Unknown connection entry 23 for command PingCommand
>>>
>>> I have closed all client connections, checked for zombies via ps, and even
>>> restarted the server to try and clear it out but nothing seems to work.
>>> According to the dba_sql_statements table there are no rogue statement
>> being
>>> processed either.
>>>
>>> What does this mean and how do stop it from filling up my logs?
>>
>> ----------------------------------------------------------------------------
>> --
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> luciddb-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>>
>
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>


------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Loading...