Quantcast

server crash during import

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

server crash during import

Christian Herta
Hello,

I have problems to import data from a mysql database. With a toy example
it works, but with a real DB (one
table with 41000000 rows) the server crashes. I am new to luciddb, so I
am not sure, if I use it  correctly. Below
you will find how I proceed and the corresponding the log messages.
 
Thanks in advance,
 Christian

-------------------------
I connect to the mysql DB via:

create or replace server mysql_test
foreign data wrapper sys_jdbc options(
driver_class 'com.mysql.jdbc.Driver',
login_timeout '10',
url 'jdbc:mysql://xxxxx:12306/annoDB',
user_name 'xxxx',
password 'xxxx',
validation_query 'select 1',
fetch_size '-2147483648',
table_types 'TABLE',
schema_name 'DEFAULT');


To import the data I use the following command (into an existing table
"annotations"):

insert into "annotations" select * from  mysql_test."DEFAULT"."annotations";

---

INFO: insert into "annotations" select * from
mysql_test."DEFAULT"."annotations"
05.10.2009 16:55:45 org.eigenbase.util.EigenbaseException <init>
SCHWERWIEGEND: org.eigenbase.util.EigenbaseException: Failed to look up
foreign table [MYSQL_TEST, DEFAULT, annotations]
05.10.2009 16:55:45 net.sf.farrago.jdbc.FarragoJdbcUtil newSqlException
SCHWERWIEGEND: Failed to look up foreign table [MYSQL_TEST, DEFAULT,
annotations]
05.10.2009 16:55:45 net.sf.farrago.jdbc.FarragoJdbcUtil newSqlException
SCHWERWIEGEND: Java heap space
05.10.2009 16:55:45 net.sf.farrago.db.FarragoDbStmtContext cancel
INFO: cancel
05.10.2009 16:55:45 net.sf.fennel.backtrace <native>
SCHWERWIEGEND: *** CAUGHT SIGNAL 6; BACKTRACE:
/daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x51)
[0x2ac41]
/lib64/libpthread.so.0 [0x7f25c9831a90]
/lib64/libc.so.6(gsignal+0x35) [0x7f25c90f3645]
/lib64/libc.so.6(abort+0x183) [0x7f25c90f4c33]
/daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__gnu_cxx::__verbose_terminate_handler()+0x114)
[0x80104]
/daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__cxxabiv1::__terminate(void
(*)())+0x6) [0x7e506]
/daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1
[0x7f25307c2533]
/daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__cxa_rethrow+0x45)
[0x7e5b5]
/daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::TimerThread::run()+0x349)
[0x56db9]
/daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::Thread::initAndRun()+0x12)
[0x55052]
/daten/installations/luciddb-0.9.1/lib/fennel/libboost_thread-mt-p.so(thread_proxy+0x5f)
[0x7f25312f1e1f]
/lib64/libpthread.so.0 [0x7f25c982a070]
/lib64/libc.so.6(clone+0x6d) [0x7f25c919410d]

------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

John Sichi
Administrator
Hi Christian,

 From the log, the cause of the crash was that the server ran out of
Java heap.  The most likely cause is the MySQL JDBC driver (running
inside the LucidDB JVM), it eats up all of the memory unless streaming
mode is used for the fetch.

http://n2.nabble.com/MED-JDBC-Memory-blowouts-td1441508.html#a1441510

If you are using the 5.x series of MySQL Connector/J, you'll need to add
the userCursorFetch=true setting to the URL as mentioned in this page:

http://pub.eigenbase.org/wiki/LucidDbCreateForeignServer#MySQL

JVS

Christian Herta wrote:

> Hello,
>
> I have problems to import data from a mysql database. With a toy example
> it works, but with a real DB (one
> table with 41000000 rows) the server crashes. I am new to luciddb, so I
> am not sure, if I use it  correctly. Below
> you will find how I proceed and the corresponding the log messages.
>  
> Thanks in advance,
>  Christian
>
> -------------------------
> I connect to the mysql DB via:
>
> create or replace server mysql_test
> foreign data wrapper sys_jdbc options(
> driver_class 'com.mysql.jdbc.Driver',
> login_timeout '10',
> url 'jdbc:mysql://xxxxx:12306/annoDB',
> user_name 'xxxx',
> password 'xxxx',
> validation_query 'select 1',
> fetch_size '-2147483648',
> table_types 'TABLE',
> schema_name 'DEFAULT');
>
>
> To import the data I use the following command (into an existing table
> "annotations"):
>
> insert into "annotations" select * from  mysql_test."DEFAULT"."annotations";
>
> ---
>
> INFO: insert into "annotations" select * from
> mysql_test."DEFAULT"."annotations"
> 05.10.2009 16:55:45 org.eigenbase.util.EigenbaseException <init>
> SCHWERWIEGEND: org.eigenbase.util.EigenbaseException: Failed to look up
> foreign table [MYSQL_TEST, DEFAULT, annotations]
> 05.10.2009 16:55:45 net.sf.farrago.jdbc.FarragoJdbcUtil newSqlException
> SCHWERWIEGEND: Failed to look up foreign table [MYSQL_TEST, DEFAULT,
> annotations]
> 05.10.2009 16:55:45 net.sf.farrago.jdbc.FarragoJdbcUtil newSqlException
> SCHWERWIEGEND: Java heap space
> 05.10.2009 16:55:45 net.sf.farrago.db.FarragoDbStmtContext cancel
> INFO: cancel
> 05.10.2009 16:55:45 net.sf.fennel.backtrace <native>
> SCHWERWIEGEND: *** CAUGHT SIGNAL 6; BACKTRACE:
> /daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x51)
> [0x2ac41]
> /lib64/libpthread.so.0 [0x7f25c9831a90]
> /lib64/libc.so.6(gsignal+0x35) [0x7f25c90f3645]
> /lib64/libc.so.6(abort+0x183) [0x7f25c90f4c33]
> /daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__gnu_cxx::__verbose_terminate_handler()+0x114)
> [0x80104]
> /daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__cxxabiv1::__terminate(void
> (*)())+0x6) [0x7e506]
> /daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1
> [0x7f25307c2533]
> /daten/installations/luciddb-0.9.1/lib/fennel/libstlport.so.5.1(__cxa_rethrow+0x45)
> [0x7e5b5]
> /daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::TimerThread::run()+0x349)
> [0x56db9]
> /daten/installations/luciddb-0.9.1/lib/fennel/libfennel_common.so(fennel::Thread::initAndRun()+0x12)
> [0x55052]
> /daten/installations/luciddb-0.9.1/lib/fennel/libboost_thread-mt-p.so(thread_proxy+0x5f)
> [0x7f25312f1e1f]
> /lib64/libpthread.so.0 [0x7f25c982a070]
> /lib64/libc.so.6(clone+0x6d) [0x7f25c919410d]
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users
>


------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

Jeremy Lemaire
My server keeps crashing during import.  From the stack traces below it appears I may be running out of memory however, I have tried various memory settings and nothing seems to stabilize the system.  The concern I have is that I don;t appear to be anywhere near the 16Gb system limit.  Currently I have reserved 4Gb for cache pages and 4Gb for Java Heap.  I have also tried 12Gb, 8Gb and 6Gb cache, and 1Gb, 2Gb Java Heap. Concurrent connections is set to 20.  Nothing seems to fix this.  

From the lucidDbServer console:

*** CAUGHT SIGNAL 6; BACKTRACE:
/home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x51) [0x2ad21]
/lib/libpthread.so.0 [0x7f418d006a80]
/lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
/lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
/lib/libpthread.so.0 [0x7f418d006a80]
/home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >, fennel::TraceLevel, stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
/home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd) [0x2ae8d]
/lib/libpthread.so.0 [0x7f418d006a80]
/lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
/lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
/lib/libpthread.so.0 [0x7f418d006a80]
/home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >, fennel::TraceLevel, stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
/home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd) [0x2ae8d]
/lib/libpthread.so.0 [0x7f418d006a80]
/lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
/lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
/lib/libpthread.so.0 [0x7f418d006a80]
/home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >, fennel::TraceLevel, stlp_std::basic_string<char, stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
/home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd) [0x2ae8d]
/lib/libpthread.so.0 [0x7f418d006a80]
/lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
/lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
/usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
[Too many errors, abort]
./bin/lucidDbServer: line 9: 32744 Segmentation fault      ${JAVA_EXEC} ${JAVA_ARGS} com.lucidera.farrago.LucidDbServer






From the hs_err_pid32744.log found in FARRAGO_HOME:


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x00007f407428c000 JavaThread "HashSessionScavenger-0" daemon [_thread_blocked, id=314, stack(0x0000000044788000,0x0000000044889000)]
  0x00007f407428a000 JavaThread "qtp1762218090-25" [_thread_blocked, id=313, stack(0x0000000044687000,0x0000000044788000)]
  0x00007f4074288000 JavaThread "qtp1762218090-24" [_thread_blocked, id=312, stack(0x0000000044586000,0x0000000044687000)]
  0x00007f4074286000 JavaThread "qtp1762218090-23" [_thread_blocked, id=311, stack(0x0000000044485000,0x0000000044586000)]
  0x00007f407404c000 JavaThread "qtp1762218090-22" [_thread_blocked, id=310, stack(0x0000000044384000,0x0000000044485000)]
  0x00007f407426e800 JavaThread "qtp1762218090-21" [_thread_blocked, id=309, stack(0x0000000044283000,0x0000000044384000)]
  0x00007f407426e000 JavaThread "qtp1762218090-20" [_thread_blocked, id=308, stack(0x0000000044182000,0x0000000044283000)]
  0x00007f407404f800 JavaThread "qtp1762218090-19" [_thread_blocked, id=307, stack(0x0000000044081000,0x0000000044182000)]
  0x00007f4074070800 JavaThread "qtp1762218090-18 - Acceptor0 SelectChannelConnector@0.0.0.0:8034" [_thread_in_native, id=306, stack(0x0000000043f80000,0x0000000044081000)]
  0x00007f4074088800 JavaThread "Timer-1" daemon [_thread_blocked, id=305, stack(0x0000000043e7f000,0x0000000043f80000)]
  0x00007f407407b000 JavaThread "Farrago Watchdog Timer" [_thread_blocked, id=304, stack(0x0000000041b7a000,0x0000000041c7b000)]
  0x00007f407bcf8800 JavaThread "Thread-4" [_thread_in_native, id=303, stack(0x000000004367e000,0x0000000043e7f000)]
  0x00007f40802f2000 JavaThread "Enki MDR Change Event Thread" daemon [_thread_blocked, id=32765, stack(0x0000000040b36000,0x0000000040c37000)]
  0x00007f4080687800 JavaThread "HSQLDB Timer @1018f51f" daemon [_thread_blocked, id=32764, stack(0x0000000041795000,0x0000000041896000)]
  0x00007f4080c6e000 JavaThread "Timer-0" daemon [_thread_blocked, id=32763, stack(0x00000000404eb000,0x00000000405ec000)]
  0x00007f4080c5b800 JavaThread "Active Reference Queue Daemon" daemon [_thread_blocked, id=32762, stack(0x0000000041694000,0x0000000041795000)]
  0x00007f407c008000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=32760, stack(0x00000000403ea000,0x00000000404eb000)]
  0x00007f4081ae4000 JavaThread "CompilerThread1" daemon [_thread_blocked, id=32759, stack(0x0000000040a35000,0x0000000040b36000)]
  0x00007f4081ae0800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=32758, stack(0x0000000041593000,0x0000000041694000)]
  0x00007f4081ade800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=32757, stack(0x00000000402e9000,0x00000000403ea000)]
  0x00007f4081abf000 JavaThread "Finalizer" daemon [_thread_blocked, id=32756, stack(0x0000000041d7a000,0x0000000041e7b000)]
  0x00007f4081abd000 JavaThread "Reference Handler" daemon [_thread_blocked, id=32755, stack(0x00000000408f5000,0x00000000409f6000)]
  0x00000000410fd000 JavaThread "main" [_thread_in_native, id=32745, stack(0x000000004140d000,0x000000004150e000)]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

John Sichi
Administrator
Please read the recent thread about configuring the MySQL JDBC driver to
make sure it is using streaming cursor fetch.

JVS

haawker wrote:

> My server keeps crashing during import.  From the stack traces below it
> appears I may be running out of memory however, I have tried various memory
> settings and nothing seems to stabilize the system.  The concern I have is
> that I don;t appear to be anywhere near the 16Gb system limit.  Currently I
> have reserved 4Gb for cache pages and 4Gb for Java Heap.  I have also tried
> 12Gb, 8Gb and 6Gb cache, and 1Gb, 2Gb Java Heap. Concurrent connections is
> set to 20.  Nothing seems to fix this.  
>
>>From the lucidDbServer console:
>
> *** CAUGHT SIGNAL 6; BACKTRACE:
> /home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x51)
> [0x2ad21]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
> /lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >,
> fennel::TraceLevel, stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
> /home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd)
> [0x2ae8d]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
> /lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >,
> fennel::TraceLevel, stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
> /home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd)
> [0x2ae8d]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
> /lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c738c4f]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c739211]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /home/lucid/luciddb-0.9.2/lib/fennel/libfarrago.so(fennel::JavaTraceTarget::notifyTrace(stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >,
> fennel::TraceLevel, stlp_std::basic_string<char,
> stlp_std::char_traits<char>, stlp_std::allocator<char> >)+0x60) [0x229e40]
> /home/lucid/luciddb/lib/fennel/libfennel_common.so(fennel::AutoBacktrace::signal_handler(int)+0x1bd)
> [0x2ae8d]
> /lib/libpthread.so.0 [0x7f418d006a80]
> /lib/libc.so.6(gsignal+0x35) [0x7f418cad2ed5]
> /lib/libc.so.6(abort+0x183) [0x7f418cad43f3]
> /usr/opt/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so [0x7f418c5ff679]
> [Too many errors, abort]
> ./bin/lucidDbServer: line 9: 32744 Segmentation fault      ${JAVA_EXEC}
> ${JAVA_ARGS} com.lucidera.farrago.LucidDbServer
>
>
>
>
>
>
>>From the hs_err_pid32744.log found in FARRAGO_HOME:
>
>
> ---------------  P R O C E S S  ---------------
>
> Java Threads: ( => current thread )
>   0x00007f407428c000 JavaThread "HashSessionScavenger-0" daemon
> [_thread_blocked, id=314, stack(0x0000000044788000,0x0000000044889000)]
>   0x00007f407428a000 JavaThread "qtp1762218090-25" [_thread_blocked, id=313,
> stack(0x0000000044687000,0x0000000044788000)]
>   0x00007f4074288000 JavaThread "qtp1762218090-24" [_thread_blocked, id=312,
> stack(0x0000000044586000,0x0000000044687000)]
>   0x00007f4074286000 JavaThread "qtp1762218090-23" [_thread_blocked, id=311,
> stack(0x0000000044485000,0x0000000044586000)]
>   0x00007f407404c000 JavaThread "qtp1762218090-22" [_thread_blocked, id=310,
> stack(0x0000000044384000,0x0000000044485000)]
>   0x00007f407426e800 JavaThread "qtp1762218090-21" [_thread_blocked, id=309,
> stack(0x0000000044283000,0x0000000044384000)]
>   0x00007f407426e000 JavaThread "qtp1762218090-20" [_thread_blocked, id=308,
> stack(0x0000000044182000,0x0000000044283000)]
>   0x00007f407404f800 JavaThread "qtp1762218090-19" [_thread_blocked, id=307,
> stack(0x0000000044081000,0x0000000044182000)]
>   0x00007f4074070800 JavaThread "qtp1762218090-18 - Acceptor0
> SelectChannelConnector@0.0.0.0:8034" [_thread_in_native, id=306,
> stack(0x0000000043f80000,0x0000000044081000)]
>   0x00007f4074088800 JavaThread "Timer-1" daemon [_thread_blocked, id=305,
> stack(0x0000000043e7f000,0x0000000043f80000)]
>   0x00007f407407b000 JavaThread "Farrago Watchdog Timer" [_thread_blocked,
> id=304, stack(0x0000000041b7a000,0x0000000041c7b000)]
>   0x00007f407bcf8800 JavaThread "Thread-4" [_thread_in_native, id=303,
> stack(0x000000004367e000,0x0000000043e7f000)]
>   0x00007f40802f2000 JavaThread "Enki MDR Change Event Thread" daemon
> [_thread_blocked, id=32765, stack(0x0000000040b36000,0x0000000040c37000)]
>   0x00007f4080687800 JavaThread "HSQLDB Timer @1018f51f" daemon
> [_thread_blocked, id=32764, stack(0x0000000041795000,0x0000000041896000)]
>   0x00007f4080c6e000 JavaThread "Timer-0" daemon [_thread_blocked, id=32763,
> stack(0x00000000404eb000,0x00000000405ec000)]
>   0x00007f4080c5b800 JavaThread "Active Reference Queue Daemon" daemon
> [_thread_blocked, id=32762, stack(0x0000000041694000,0x0000000041795000)]
>   0x00007f407c008000 JavaThread "Low Memory Detector" daemon
> [_thread_blocked, id=32760, stack(0x00000000403ea000,0x00000000404eb000)]
>   0x00007f4081ae4000 JavaThread "CompilerThread1" daemon [_thread_blocked,
> id=32759, stack(0x0000000040a35000,0x0000000040b36000)]
>   0x00007f4081ae0800 JavaThread "CompilerThread0" daemon [_thread_blocked,
> id=32758, stack(0x0000000041593000,0x0000000041694000)]
>   0x00007f4081ade800 JavaThread "Signal Dispatcher" daemon [_thread_blocked,
> id=32757, stack(0x00000000402e9000,0x00000000403ea000)]
>   0x00007f4081abf000 JavaThread "Finalizer" daemon [_thread_blocked,
> id=32756, stack(0x0000000041d7a000,0x0000000041e7b000)]
>   0x00007f4081abd000 JavaThread "Reference Handler" daemon [_thread_blocked,
> id=32755, stack(0x00000000408f5000,0x00000000409f6000)]
>   0x00000000410fd000 JavaThread "main" [_thread_in_native, id=32745,
> stack(0x000000004140d000,0x000000004150e000)]
>
>


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

Jeremy Lemaire
This post was updated on .
I have read this but was not sure that it applied given I am not using the MySQL JDBC driver.  Does this work around apply to the foreign data wrappers I have configured (please see below)?

-----------------------------------------
-- Foreign Data Wrapper Definitions
-----------------------------------------

--drop server ad_server_file_link cascade;
--drop server adsdw01_psrv cascade;

-----------------------------------------
-- Imported CSV Data
-----------------------------------------
create server ad_server_file_link
foreign data wrapper sys_file_wrapper
options (
    directory '${FARRAGO_HOME}/import',
    file_extension 'csv',
    with_header 'NO', 
    log_directory 'trace/',
    field_delimiter ',',
    timestamp_format 'yyyy-MM-dd HH:mm:ss' );
 
-----------------------------------------
-- adsdw01 Local Data
-----------------------------------------   
create server adsdw01_psrv
foreign data wrapper sys_jdbc
options(
    driver_class 'com.lucidera.jdbc.LucidDbLocalDriver',
    url 'jdbc:luciddb:',
    user_name 'sa',
    password '*********'
);
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

John Sichi
Administrator
What is the command which is crashing?

JVS

haawker wrote:

> I have read this but was not sure that it applied given I am not using the
> MySQL JDBC driver.  Does this work around apply to the foreign data wrappers
> I have configured (please see below)?
>
> -----------------------------------------
> -- Foreign Data Wrapper Definitions
> -----------------------------------------
>
> --drop server ad_server_file_link cascade;
> --drop server adsdw01_psrv cascade;
>
> -----------------------------------------
> -- Imported CSV Data
> -----------------------------------------
> create server ad_server_file_link
> foreign data wrapper sys_file_wrapper
> options (
>     directory '${FARRAGO_HOME}/import',
>     file_extension 'csv',
>     with_header 'NO',
>     log_directory 'trace/',
>     field_delimiter ',',
>     timestamp_format 'yyyy-MM-dd HH:mm:ss' );
>  
> -----------------------------------------
> -- adsdw01 Local Data
> -----------------------------------------  
> create server adsdw01_psrv
> foreign data wrapper sys_jdbc
> options(
>     driver_class 'com.lucidera.jdbc.LucidDbLocalDriver',
>     url 'jdbc:luciddb:',
>     user_name 'sa',
>     password 'lucid4now'
> );


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

Jeremy Lemaire
It is the insert into ad_inventory_fact_2009_q2 command in this case, but it varies.  It seems to be more time (or memory) dependent than any one particular command.  Generally I see it on big inserts though (i.e. >20M rows).   Running the following for a months worth of data worked for a couple months, then had to be restarted 3 times for one month, then ran for another couple months.


create foreign table ad_server_extraction_schema."ad_inventory" (
 	ad_inventory_id int,
 	source_id int,                  
 	caller_id varchar(32),            
 	reason int,
 	ts timestamp,
 	age varchar(32),
 	gender varchar(32),
 	"LANGUAGE" varchar(32),
 	phone_type varchar(32),
 	carrier varchar(32),
 	priority int,      
 	country varchar(32),    
 	zipcode varchar(32),        
 	areacode varchar(32),    
 	dma varchar(32),           
 	income varchar(32),         
 	genre varchar(32),         
 	extra_params varchar(128), 
 	ad_run_id int, 
 	call_id bigint,   
 	state varchar(32),        
 	destination varchar(32),      
 	session_id varchar(32),       
 	ip_address varchar(32),     
 	email varchar(32),        
 	username varchar(32) )      
server ad_server_file_link;

create view ad_inventory_union_view as
select source_id, reason, ts, ad_run_id, call_id,
     CASE
          WHEN inv.age IS NULL THEN 'unknown'
          ELSE inv.age
     END as age,
     CASE
          WHEN inv.gender IS NULL THEN 'unknown'
          ELSE inv.gender
     END as gender,
     CASE
          WHEN inv."LANGUAGE" IS NULL THEN 'unknown'
          ELSE inv."LANGUAGE"
     END as "LANGUAGE",
     CASE
          WHEN inv.phone_type IS NULL THEN 'unknown'
          ELSE inv.phone_type
     END as phone_type,
     CASE
          WHEN inv.carrier IS NULL THEN 'unknown'
          ELSE inv.carrier
     END as carrier,
     CASE
          WHEN inv.country IS NULL THEN 'unknown'
          ELSE inv.country
     END as country,
     CASE
          WHEN inv.zipcode IS NULL THEN 'unknown'
          ELSE inv.zipcode
     END as zipcode,
     CASE
          WHEN inv.areacode IS NULL THEN 'unknown'
          ELSE inv.areacode
     END as areacode,
     CASE
          WHEN inv.dma IS NULL THEN 'unknown'
          ELSE inv.dma
     END as dma,
     CASE
          WHEN inv.income IS NULL THEN 'unknown'
          ELSE inv.income
     END as income,
     CASE
          WHEN inv.genre IS NULL THEN 'unknown'
          ELSE inv.genre
     END as genre,
     CASE
          WHEN inv.extra_params IS NULL THEN 'unknown'
          ELSE inv.extra_params
     END as extra_params,
     CASE
          WHEN inv.state IS NULL THEN 'unknown'
          ELSE inv.state
     END as state,
     CASE
          WHEN inv.destination IS NULL THEN 'unknown'
          ELSE inv.destination
     END as destination, 
     CASE
          WHEN inv.caller_id IS NULL THEN 'unknown'
          ELSE inv.caller_id
     END as caller_id,
     CASE
          WHEN inv.ip_address IS NULL THEN 'unknown'
          ELSE inv.ip_address
     END as ip_address, 
     CASE
          WHEN inv.email IS NULL THEN 'unknown'
          ELSE inv.email
     END as email, 
     CASE
          WHEN inv.username IS NULL THEN 'unknown'
          ELSE inv.username
     END as username 
from ad_server_extraction_schema."ad_inventory" inv;


create view ad_server_transform_schema.ad_inventory_view as
select source_id, reason, ts, ad_run_id, call_id, caller_id, ip_address, email, username, parameter_set_key from ad_server_transform_schema.ad_inventory_union_view inv
    left outer join parameter_set_dim on 
            (inv.age=parameter_set_dim.age)   and
            (inv.gender=parameter_set_dim.gender)  and
            (inv."LANGUAGE"=parameter_set_dim."LANGUAGE")  and
            (inv.phone_type=parameter_set_dim.phone_type)  and
            (inv.carrier=parameter_set_dim.carrier)  and
            (inv.country=parameter_set_dim.country)  and
            (inv.zipcode=parameter_set_dim.zipcode)  and
            (inv.areacode=parameter_set_dim.areacode)  and
            (inv.dma=parameter_set_dim.dma)  and
            (inv.income=parameter_set_dim.income)  and
            (inv.genre=parameter_set_dim.genre)  and
            (inv.state=parameter_set_dim.state)  and
            (inv.destination=parameter_set_dim.destination);


insert into ad_inventory_fact_2009_q2(
    source_id,
    reason,
    ts,
    ad_run_id,
    call_id,
    caller_id,
    ip_address,
    email,
    username,
    parameter_set_id)
select
    source_id,
    reason,
    ts,
    ad_run_id,
    call_id,
    caller_id,
    ip_address,
    email,
    username,
    parameter_set_key
from ad_server_transform_schema.ad_inventory_view;
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

John Sichi
Administrator
Sounds like a leak.  I'd have to see a Java heap dump after it had been
running for a while to say more.

You might try

select * from sys_root.dba_sql_statements;

and

select * from sys_root.dba_sessions;

to see if either of those are accumulating unexpectedly.

JVS

haawker wrote:

> It is the insert into ad_inventory_fact_2009_q2 command in this case, but it
> varies.  It seems to be more time (or memory) dependent than any one
> particular command.  Generally I see it on big inserts though (i.e. >20M
> rows).   Running the following for a months worth of data worked for a
> couple months, then had to be restarted 3 times for one month, then ran for
> another couple months.
>
>
> create foreign table ad_server_extraction_schema."ad_inventory" (
>   ad_inventory_id int,
>   source_id int,                  
>   caller_id varchar(32),            
>   reason int,
>   ts timestamp,
>   age varchar(32),
>   gender varchar(32),
>   "LANGUAGE" varchar(32),
>   phone_type varchar(32),
>   carrier varchar(32),
>   priority int,      
>   country varchar(32),    
>   zipcode varchar(32),        
>   areacode varchar(32),    
>   dma varchar(32),          
>   income varchar(32),        
>   genre varchar(32),        
>   extra_params varchar(128),
>   ad_run_id int,
>   call_id bigint,  
>   state varchar(32),        
>   destination varchar(32),      
>   session_id varchar(32),      
>   ip_address varchar(32),    
>   email varchar(32),        
>   username varchar(32) )      
> server ad_server_file_link;
>
> create view ad_inventory_union_view as
> select source_id, reason, ts, ad_run_id, call_id,
>      CASE
>           WHEN inv.age IS NULL THEN 'unknown'
>           ELSE inv.age
>      END as age,
>      CASE
>           WHEN inv.gender IS NULL THEN 'unknown'
>           ELSE inv.gender
>      END as gender,
>      CASE
>           WHEN inv."LANGUAGE" IS NULL THEN 'unknown'
>           ELSE inv."LANGUAGE"
>      END as "LANGUAGE",
>      CASE
>           WHEN inv.phone_type IS NULL THEN 'unknown'
>           ELSE inv.phone_type
>      END as phone_type,
>      CASE
>           WHEN inv.carrier IS NULL THEN 'unknown'
>           ELSE inv.carrier
>      END as carrier,
>      CASE
>           WHEN inv.country IS NULL THEN 'unknown'
>           ELSE inv.country
>      END as country,
>      CASE
>           WHEN inv.zipcode IS NULL THEN 'unknown'
>           ELSE inv.zipcode
>      END as zipcode,
>      CASE
>           WHEN inv.areacode IS NULL THEN 'unknown'
>           ELSE inv.areacode
>      END as areacode,
>      CASE
>           WHEN inv.dma IS NULL THEN 'unknown'
>           ELSE inv.dma
>      END as dma,
>      CASE
>           WHEN inv.income IS NULL THEN 'unknown'
>           ELSE inv.income
>      END as income,
>      CASE
>           WHEN inv.genre IS NULL THEN 'unknown'
>           ELSE inv.genre
>      END as genre,
>      CASE
>           WHEN inv.extra_params IS NULL THEN 'unknown'
>           ELSE inv.extra_params
>      END as extra_params,
>      CASE
>           WHEN inv.state IS NULL THEN 'unknown'
>           ELSE inv.state
>      END as state,
>      CASE
>           WHEN inv.destination IS NULL THEN 'unknown'
>           ELSE inv.destination
>      END as destination,
>      CASE
>           WHEN inv.caller_id IS NULL THEN 'unknown'
>           ELSE inv.caller_id
>      END as caller_id,
>      CASE
>           WHEN inv.ip_address IS NULL THEN 'unknown'
>           ELSE inv.ip_address
>      END as ip_address,
>      CASE
>           WHEN inv.email IS NULL THEN 'unknown'
>           ELSE inv.email
>      END as email,
>      CASE
>           WHEN inv.username IS NULL THEN 'unknown'
>           ELSE inv.username
>      END as username
> from ad_server_extraction_schema."ad_inventory" inv;
>
>
> create view ad_server_transform_schema.ad_inventory_view as
> select source_id, reason, ts, ad_run_id, call_id, caller_id, ip_address,
> email, username, parameter_set_key from
> ad_server_transform_schema.ad_inventory_union_view inv
>     left outer join parameter_set_dim on
>             (inv.age=parameter_set_dim.age)   and
>             (inv.gender=parameter_set_dim.gender)  and
>             (inv."LANGUAGE"=parameter_set_dim."LANGUAGE")  and
>             (inv.phone_type=parameter_set_dim.phone_type)  and
>             (inv.carrier=parameter_set_dim.carrier)  and
>             (inv.country=parameter_set_dim.country)  and
>             (inv.zipcode=parameter_set_dim.zipcode)  and
>             (inv.areacode=parameter_set_dim.areacode)  and
>             (inv.dma=parameter_set_dim.dma)  and
>             (inv.income=parameter_set_dim.income)  and
>             (inv.genre=parameter_set_dim.genre)  and
>             (inv.state=parameter_set_dim.state)  and
>             (inv.destination=parameter_set_dim.destination);
>
>
> insert into ad_inventory_fact_2009_q2(
>     source_id,
>     reason,
>     ts,
>     ad_run_id,
>     call_id,
>     caller_id,
>     ip_address,
>     email,
>     username,
>     parameter_set_id)
> select
>     source_id,
>     reason,
>     ts,
>     ad_run_id,
>     call_id,
>     caller_id,
>     ip_address,
>     email,
>     username,
>     parameter_set_key
> from ad_server_transform_schema.ad_inventory_view;
>


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

Jeremy Lemaire
There are no statements accumulating, in fact the last time this happened (2am this morning) there was only one insert and one select running.  You mentioned that you may be able to say more if you saw the Java Heap.  The following was taken from the hs_err_pidxxxx.log.  Will this suffice?


Heap
 PSYoungGen      total 174272K, used 174240K [0x00007f0a2e470000, 0x00007f0a38f10000, 0x00007f0a839c0000)
  eden space 173952K, 100% used [0x00007f0a2e470000,0x00007f0a38e50000,0x00007f0a38e50000)
  from space 320K, 90% used [0x00007f0a38ec0000,0x00007f0a38f08000,0x00007f0a38f10000)
  to   space 384K, 75% used [0x00007f0a38e50000,0x00007f0a38e98000,0x00007f0a38eb0000)
 PSOldGen        total 2358336K, used 2321076K [0x00007f09839c0000, 0x00007f0a138d0000, 0x00007f0a2e470000)
  object space 2358336K, 98% used [0x00007f09839c0000,0x00007f0a1146d3c8,0x00007f0a138d0000)
 PSPermGen       total 86016K, used 78258K [0x00007f097e5c0000, 0x00007f09839c0000, 0x00007f09839c0000)
  object space 86016K, 90% used [0x00007f097e5c0000,0x00007f098322c988,0x00007f09839c0000)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

John Sichi
Administrator
Hey Jeremy,

We did indeed find a leak very recently:

http://issues.eigenbase.org/browse/FNL-89

It may or may not be what is causing your crashes; it showed up in a
long-running server where the same statements were being executed
repeatedly.

It is fixed on the latest trunk, but this has not yet been released; to
test it out, you can build it yourself, or if you let me know your
platform, I can build a prerelease binary, or if you want an isolated
patch for 0.9.2, Dynamo BI is providing paid commercial support.

JVS

haawker wrote:

> There are no statements accumulating, in fact the last time this happened
> (2am this morning) there was only one insert and one select running.  You
> mentioned that you may be able to say more if you saw the Java Heap.  The
> following was taken from the hs_err_pidxxxx.log.  Will this suffice?
>
>
> Heap
>  PSYoungGen      total 174272K, used 174240K [0x00007f0a2e470000,
> 0x00007f0a38f10000, 0x00007f0a839c0000)
>   eden space 173952K, 100% used
> [0x00007f0a2e470000,0x00007f0a38e50000,0x00007f0a38e50000)
>   from space 320K, 90% used
> [0x00007f0a38ec0000,0x00007f0a38f08000,0x00007f0a38f10000)
>   to   space 384K, 75% used
> [0x00007f0a38e50000,0x00007f0a38e98000,0x00007f0a38eb0000)
>  PSOldGen        total 2358336K, used 2321076K [0x00007f09839c0000,
> 0x00007f0a138d0000, 0x00007f0a2e470000)
>   object space 2358336K, 98% used
> [0x00007f09839c0000,0x00007f0a1146d3c8,0x00007f0a138d0000)
>  PSPermGen       total 86016K, used 78258K [0x00007f097e5c0000,
> 0x00007f09839c0000, 0x00007f09839c0000)
>   object space 86016K, 90% used
> [0x00007f097e5c0000,0x00007f098322c988,0x00007f09839c0000)
>


------------------------------------------------------------------------------

_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: server crash during import

ngoodman
FNL-89 comment has a jdk 1.6 method to determine if this is the issue  
that is causing you problems (jmap | grep).  The Jira has the .patch  
and this will certainly be fixed with 0.9.3; we've already built this  
patched release for our customers.

Starting / stopping LucidDB regularly is another straightforward  
workaround.

Nick

On May 27, 2010, at 11:33 PM, "John V. Sichi" <[hidden email]> wrote:

> Hey Jeremy,
>
> We did indeed find a leak very recently:
>
> http://issues.eigenbase.org/browse/FNL-89
>
> It may or may not be what is causing your crashes; it showed up in a
> long-running server where the same statements were being executed
> repeatedly.
>
> It is fixed on the latest trunk, but this has not yet been released;  
> to
> test it out, you can build it yourself, or if you let me know your
> platform, I can build a prerelease binary, or if you want an isolated
> patch for 0.9.2, Dynamo BI is providing paid commercial support.
>
> JVS
>
> haawker wrote:
>> There are no statements accumulating, in fact the last time this  
>> happened
>> (2am this morning) there was only one insert and one select  
>> running.  You
>> mentioned that you may be able to say more if you saw the Java  
>> Heap.  The
>> following was taken from the hs_err_pidxxxx.log.  Will this suffice?
>>
>>
>> Heap
>> PSYoungGen      total 174272K, used 174240K [0x00007f0a2e470000,
>> 0x00007f0a38f10000, 0x00007f0a839c0000)
>>  eden space 173952K, 100% used
>> [0x00007f0a2e470000,0x00007f0a38e50000,0x00007f0a38e50000)
>>  from space 320K, 90% used
>> [0x00007f0a38ec0000,0x00007f0a38f08000,0x00007f0a38f10000)
>>  to   space 384K, 75% used
>> [0x00007f0a38e50000,0x00007f0a38e98000,0x00007f0a38eb0000)
>> PSOldGen        total 2358336K, used 2321076K [0x00007f09839c0000,
>> 0x00007f0a138d0000, 0x00007f0a2e470000)
>>  object space 2358336K, 98% used
>> [0x00007f09839c0000,0x00007f0a1146d3c8,0x00007f0a138d0000)
>> PSPermGen       total 86016K, used 78258K [0x00007f097e5c0000,
>> 0x00007f09839c0000, 0x00007f09839c0000)
>>  object space 86016K, 90% used
>> [0x00007f097e5c0000,0x00007f098322c988,0x00007f09839c0000)
>>
>
>
> ---
> ---
> ---
> ---------------------------------------------------------------------
>
> _______________________________________________
> luciddb-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/luciddb-users

------------------------------------------------------------------------------

_______________________________________________
luciddb-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/luciddb-users
Loading...