LucidDB Gets Stuck Processing Many Concurrent Queries

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

LucidDB Gets Stuck Processing Many Concurrent Queries

victor.savkin
Hi Guys,

I have a heavy query. I'm trying to run this query in parallel using X threads.

It works fine in 5 threads.
It works fine in 7 threads.

But each time I'm running it in 8 threads I got 7 responses but 1 thread always gets stuck. I don't get any answer from luciddb. If I run all 8 queries with a small random delay, so they won't be started at the same time it works fine. Could you help me to understand what is causing the issue?

LucidDB configuration:
expectedConcurrentStatements is 12

Thanks,
Victor
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: LucidDB Gets Stuck Processing Many Concurrent Queries

Nicholas Goodman

On May 31, 2011, at 2:18 PM, victor.savkin wrote:

> I have a heavy query. I'm trying to run this query in parallel using X
> threads.
How are you trying to do parallel?  Are they all separate connections, queries that you are running separately or are you using a UNIONed dataset with degreeOfParallelism?

> But each time I'm running it in 8 threads I got 7 responses but 1 thread
> always gets stuck. I don't get any answer from luciddb. If I run all 8
> queries with a small random delay, so they won't be started at the same time
> it works fine. Could you help me to understand what is causing the issue?
Can you determine if all 8 queries are being executed (check the Trace.log for the statement logging).

I know of no artificial limit to 8 threads.  If one is being run, but is stuck you should also see it in http://pub.eigenbase.org/wiki/LucidDbSystemViews#DBA_SQL_STATEMENTS

Can you confirm that you see 8 records for your queries in there, and that 7 of the 8 return successfully (disappear) but that one remains?


------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: LucidDB Gets Stuck Processing Many Concurrent Queries

victor.savkin
This post was updated on .
Hi Nicholas,

Thank you for your response.  Answering your questions:
1. I created 8 separate connections.
2. I don't think that there is a limit to 8 threads somewhere. I can run another query using 12 parallel threads and everything works fine.
3. 7 queries disappear from DBA_SQL_STATEMENTS but one is there. For each thread it takes about 10 seconds to perform the query except the one that gets stuck (I waited for 30 minutes, no result).

I assume it could be a memory related issue. When I run the same query against bigger dataset two threads get stuck. When I run it against smaller dataset all 8 threads work fine.

Thanks,
Victor
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: LucidDB Gets Stuck Processing Many Concurrent Queries

ngoodman
On Jun 1, 2011, at 9:00 AM, victor.savkin wrote:

> 3. 7 queries disappear from DBA_SQL_STATEMENTS but one is there. For each
> thread it takes about 10 seconds to perform the query except the one that
> gets stuck (I waited for 10 minutes, no result).


Interesting... I've not observed this behavior before.  Are the queries identical?  Can you share their contents?

Nick

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: LucidDB Gets Stuck Processing Many Concurrent Queries

victor.savkin
This is the query I use:

select
  oi.order_id,
  oi.trnx_type,
  oi.branch_id,
  pc.category,
  pr.product_description

from
  svv.order_item oi,
  svv.order_header oh,
  svv.prod_cat pc,
  svv.product pr

where
      oi.prod_code = pr.prod_code and oi.branch_id = pr.branch_id
  and oi.prod_code = pc.prod_code and oi.branch_id = pc.branch_id
  and oi.order_id = oh.order_id and oi.branch_id = oh.branch_id
  and oh.modified_date between TIMESTAMP'2010-01-01 00:00:00' and TIMESTAMP'2010-02-01 00:00:00'


I investigated a bit further and I have more information. All threads are able to get a result set (which is about 33000 rows). 7 threads can iterate over the result set. But the 8th thread can only iterate over 5000 (each time it's a bit different) rows and than it gets blocked on resultSet.next(). Killing the application doesn't help. The session remains open.

Thanks,
Victor


Loading...